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1 . 0 PURPOSE 

The  Intelligent  Systems  Division  (ISD)  of  the  Manufacturing 
Engineering  Laboratory  (MEL)  at  the  National  Institute  of 
Standards  and  Technology  (NIST)  is  committed  to  the 
standardization  of  protocols  for  the  application  of  information 
and  interface  technology  for  the  virtual  and  distributed 
manufacturing  of  discrete  parts.  As  part  of  this  effort,  ISD  is 
developing  an  initiative  in  human  interfaces  in  manufacturing, 
emphasizing  the  integration  of  human  factors,  virtual  reality, 
and  manufacturing  systems  technologies.  A one-day  workshop, 
sponsored  by  ISD,  was  held  on  9 August  1994  to  examine  the 
application  of  information  and  interface  technology  for  the 
virtual  and  distributed  manufacturing  of  discrete  parts. 

The  purpose  of  this  report  is  to  document  the  workshop, 
including  presentations,  participant  discussions,  and  the  results 
of  working  group  deliberations.  A videotape  of  the  workshop  is 
available  at  NIST. 

2 . 0 BACKGROUND 

Manufacturing  technology  is  currently  a focus  of  national 
initiatives  sponsored  by  numerous  government  organizations  and 
industrial  consortia.  It  is  supported  by  programs  in  the 
Department  of  Commerce  (Advanced  Technology  Program  [ATP]), 
Department  of  Defense  (Technology  Reinvestment  Program  [TRP]),  as 
well  as  the  Department  of  Energy,  NASA,  and  other  Federal  and 
state  agencies.  All  of  these  programs  address  the  necessity  of 
boosting  US  manufacturing  capabilities. 

In  this  regard  NIST  has  been  given  an  expanded  role  in 
standardizing  technological  infrastructures  and  applications  in 
such  programs  as: 

* Manufacturing  Research  Technology  Centers  (METCs) , which  have 
become  major  regional  technology  resources 

* The  Automated  Manufacturing  Research  Facility  (AMRF) , an  open 
architecture  testbed  for  the  automated  manufacture  of  discrete 
parts 

* The  PDES-STEP  (Product  Data  Exchange  using  STEP  - Standard  for 
the  Exchange  of  Product  Model  Data)  Program  for  standardizing 
manufacturing  data  representations. 

2 . 1 Virtual  Reality 

Virtual  reality  (VR)  is  rapidly  becoming  technically  and 
economically  feasible.  VR  has  many  definitions  and  formats.  At 
the  ambitious  end  of  the  spectrum,  VR  allows  a user  to  have  the 
illusion  that  he  is  immersed  in  a synthetic  or  remote 
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environment.  He  experiences  sensory  information,  such  as  three- 
dimensional  vision  and  binaural  sound,  that  is  generated  by  a 
computer  or  originates  from  remote  sensors  (or  a combination  of 
the  two  sources) . He  interacts  kinesthetically  with  computer- 
generated or  teleoperated  objects.  At  the  less  ambitious  end  of 
the  spectrum,  VR  allows  a user  to  operate  in  a text-based 
interactive  mode  (i.e.,  on-line  Dungeons  and  Dragons)  or  a 
graphics-based  mode  to  manipulate  objects,  or  to  interact  with  an 
environment,  as  displayed  in  two  dimensions  on  a monitor 
(computer  or  workstation)  or  on  a head-mounted-display.  For 
manufacturing  applications,  the  modest  or  ambitious  versions  of 
VR  are  appropriate. 

The  Intelligent  Systems  Division  and  the  Factory  Automation 
Systems  Division  (FASD)  of  the  Manufacturing  Engineering 
Laboratory  at  NIST  are  concerned  with  standardizing  information 
protocols  and  interface  technologies  and  techniques  for 
manufacturing  technology.  ISD  is  examining  the  human  factors 
involved  in  interfaces  for  appropriate  information  interactions. 
NIST  is  interested  in  exploring  virtual  and  distributed 
manufacturing  with  initial  testbed  efforts  in  the  production  of 
discrete  parts.  Primary  target  areas  include:  Engineering 
Design,  Product  Planning,  Process  Planning,  Factory  Simulation, 
Monitoring  and  Control. 

ISD  is  working  with  users  and  developers  to  coordinate 
standard  information  protocols  and  interfaces,  which  will 
eventually  be  reflected  in  the  integration  of  virtual  reality  in 
a manufacturing  framework.  This  human  factors  initiative  will 
have  a major  impact  in:  information  content  for  each  user; 
presentation  bandwidth  and  fidelity;  presentation  modality  (real 
time  or  non-real  time  and  mixes  of  text,  graphics,  pictures, 
videos,  and  other  sensory  modalities) . 

Virtual  reality  may  be  able  to  integrate  intelligent 
manufacturing  systems:  products,  processes,  and  enterprises. 

The  potential  impact  of  virtual  reality  may  be  dramatic,  but  it 
will  be  defining  itself  from  the  user  pull  and  the  technology 
push  over  the  next  several  years.  VR  has  several  common  threads: 

* Virtual  reality  provides  new  worlds  for  one  or  more  users. 
Related  terminology  is:  artificial  worlds,  virtual 
environments,  and  synthetic  environments. 

* Some  VR  systems  attempt  to  give  the  user  a sense  of 
telepresence  by  immersive  sensation  (vision,  sound,  tactile 
[pressure],  haptile  [force]).  Virtual  reality  may  apply  to 
single  sites  with  human/machine  interfaces  to  co-located 
computers,  or  to  interconnected,  distributed  systems  at  distant 
sites  linked  through  communications  networks. 

* Virtual  reality  establishes  a framework  which  will  ultimately 
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integrate  many  areas  of  technology:  hiiman  factors,  computer 
hardware  and  software,  and  communications. 

* Virtual  reality  permits  each  user  to  participate  in  a 
collective  effort. 

Virtual  reality  is  supported  by  enabling  technology  programs 
in  the  following  areas: 

* Communications:  the  National  Information  Infrastructure  (Nil) 

* High  performance  computation:  High  Performance  Computing  and 
Communications  (HPCC) 

* Data  and  interface  standards:  PDES-STEP 

* Hardware  including  head  mounted  displays  (HMDs) , body  suits, 
exoskeletons,  data  gloves,  large  screens 

* Human  factors  studies. 

VR  to  date  has  been  focused  primarily  in  four  application 
areas: 

(1)  Simulation  - which  yields  new  views  and  perspectives  which 
are  unattainable  by  current  methods. 

(2)  Teleconferencing  participation  - for  "meeting  at  a distance,” 
with  a sense  of  telepresence,  in  a collaborative,  immersive 
environment  for  joint  activities,  including  discussions, 
interviews,  planning,  and  training 

(3)  Training  and  Education,  including: 

* Integration  of  training  for  personnel  at  distributed  sites 
with  distributed  elements 

* Using  simulations  or  synthetic  environments  to  achieve  cost 
savings  by  not  using  "real”  elements;  when  "real”  elements  are 
unavailable;  and  to  protect  personnel  in  training  for  hazardous 
operations  and  environments,  e.g.,  nuclear  environments,  toxic 
contamination,  or  live-fire  conditions 

* Students  using  educational  virtual  reality  toolsets  to  build 
their  own  virtual  worlds,  to  immerse  themselves,  and  to 
interact  in  "what  if?”  situations 

* Students  can  learn  by  interacting  with  unavailable  and  out-of- 
size  components  and  environments,  such  as  observing  a real-time 
manufacturing  process  through  a magnifying  glass 

(4)  Mutual  participation  in  shared  activities  in  a synthetic 
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environment.  The  most  striking  current  participatory 
applications  of  virtual  reality  are  for:  Entertainment  (several 
theme  parks  have  simulator  rides  and  other  are  currently  being 
design  as  entire  virtual  reality  theme  parks) ; and  Military  (to 
test  new  technology  or  tactics,  and  to  train  personnel  at 
distributed  sites  in  synthetic  environments) . 

2 . 2 Survey 

A survey  was  conducted  prior  to  the  workshop  to  examine  the 
human  factors  aspects  of  information  and  interfaces  needed  to 
integrate  virtual  reality  technologies  into  the  virtual  and 
distributed  manufacturing  of  discrete  parts.  The  respondents, 
who  were  experts  in  human  factors,  virtual  reality,  and 
manufacturing,  were  asked  to  address:  (1)  a definition  of  the 
requisite  information  and  interfaces  for  virtual  and  distributed 
manufacturing  applications;  (2)  human  factors  approaches  in 
information  and  interface  technologies;  and  (3)  leveraging  of 
efforts  in  specific  topical  areas.  The  results  of  the  enquiry 
will  assist  the  ISD  in  determining  the  needed  standards  and 
protocols  for  information  and  interfaces. 

The  Survey  Form  was  included  as  part  of  the  workshop 
invitation  package,  although  some  respondents  did  not  attend  the 
workshop.  The  invitation  package,  reproduced  in  Appendix  I, 
includes:  the  agenda,  workshop  objectives,  breakout  session 

topics,  registration  form,  direction  and  map  to  the  Gaithersburg 
Hilton,  background  information  for  the  survey,  and  the  enquiry 
form  (Application  of  Information  and  Interface  Technology  for 
Virtual  and  Distributed  Manufacturing  of  Discrete  Parts) . 

The  results  of  the  survey  was  delivered  to  the  ISD  as  a 
report  on  Activities  Preliminary  To  Convening  A Workshop  On  The 
Application  Of  Virtual  Reality  To  Manufacturing:  Results  Of  A 
Survey,  dated  August  94.  This  report  is  reproduced  in  its 
entirety  in  Appendix  II,  including  the  list  of  survey 
respondents,  which  is  located  in  Appendix  B (of  the  report  in 
Appendix  II) . The  workshop  attendees  are  listed  in  Appendix  III. 


3.0  A PRECIS  OF  THE  WORKSHOP 

The  following  description  of  the  Workshop  paraphrases  the 
various  presenters  and  participants  in  an  attempt  to  summarize 
the  key  issues  discussed. 

3.1  Dr.  Ernie  Kent 

Dr.  Ernie  Kent  of  the  Intelligent  Systems  Division  of  NIST, 
welcomed  the  attendees  at  the  Workshop  and  addressed  several 
issues,  as  shown  in  the  charts  in  Appendix  IV.  He  noted  that  the 
attendees  probably  have  the  following  questions:  (1)  Why  are  we 
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doing  this?  (2)  Why  are  we  here?  and  (3)  What's  in  it  for  me? 

Why  are  we  doing  this?  NIST  is  sponsoring  this  workshop  in 
an  attempt  to  identify  some  of  the  issues  in  a program  concerning 
operator  interfaces  to  virtual  distributed  manufacturing  systems. 
This  is  part  of  a larger  NIST  program.  Essentially  we  see  a 
developing  interest  in  the  intersection  of  new  technologies  for 
display,  for  human  interaction  with  traditional  kinds  of 
manufacturing  methods.  We  want  to  maintain  competence  and  assist 
industry  in  this  area.  Also,  this  is  part  of  the  traditional 
NIST  role.  In  order  to  do  this,  we  are  seeking  the  guidance  of 
experts  in  the  technology  and  in  the  industry  to  try  and  learn  to 
best  fulfill  that  mission.  That  is  essentially  why  we  are 
running  the  workshop. 

Why  are  we  here?  We  basically  tried  to  identify  the  overlap 
of  three  different  significant  areas  of  expertise:  (1)  virtual 
reality  technology:  what  technology  is  available  today,  what 
technology  will  be  available  in  the  near-term  future  that  we  can 
apply  to  operator  interfaces;  (2)  human  factors  technology:  what 
are  the  human  factors  involved  in  the  interface  display,  and  how 
do  we:  make  it  most  effective;  maximize  the  bandwidth  to  the 
human  operator;  and  present  information  with  whatever 
technologies  we  have;  and  finally,  of  course,  (3)  manufacturing 
technology:  we  need  input  from  the  manufacturing  experts  to  know 
what  kinds  of  data  should  come  across  the  interface.  This  is  not 
an  obvious  question  because  it  is  not  clear  that  the  expert  who 
runs  something  today  (when  he  is  physically  present)  really 
knows,  or  can  tell  us,  exactly  the  kind  of  information  he  needs 
or  how  he  gets  it.  We  need  to  pursue  this  in  order  to  understand 
what  we  are  doing.  The  objective  is  to  bring  together  a group  of 
fairly  disparate  people  from  different  disciplines.  I think  we 
have  a very  interesting  group  today,  with  true  experts  in  all  of 
the  areas  of  interest.  All  of  you  can  contribute  to  a dialogue 
out  of  which  we  hope  to  identify  an  intersection  of  all  of  the 
relevant  technologies. 

What's  in  it  for  all  of  us?  We  can  learn  something  from  one 
another.  Perhaps  we  can  form  a community  of  interest  to  guide 
NIST,  in  this  program,  into  the  future.  We  want  to  get  to  know 
you,  we  want  to  ask  questions,  to  learn  from  you,  and,  we  hope, 
to  have  you  communicate  with  one  another  as  this  field  develops. 
We  want  you  to  steer  NIST  to  your  benefit.  We  are  here  to  help 
you,  but,  in  order  to  help,  we  need  to  understand  the  industry 
pull  in  order  to  understand  what  we  ought  to  be  doing.  There  are 
many  NIST  programs  which  can  be  of  benefit  to  many  different 
kinds  of  organizations  and  industries,  and  it  would  be  useful  to 
maintain  cognizance  of  those. 

Let  me  tell  you,  briefly,  how  we  view  the  program  at  this 
point,  so  it  will  give  you  a basis  for  telling  us  how  to  change 
it.  We  perceive  a variety  of  industry  needs  when  we  begin  to 
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look  at  semi -automated  manufacturing  systems,  at  manufacturing 
systems  that  are  real,  but  distributed.  We  see  opportunities  in: 
product  design,  process  planning,  remote  supervision,  simulated 
”what  if”  decisions,  discussions  among  remote  experts,  training 
remotely,  and  remote  trouble  shooting. 

If  we  look  at  what  is  going  to  be  required  to  meet  those 
needs,  we  are  currently  thinking  of  something  that  looks 
approximately  like  this  [please  see  the  diagram  in  Appendix  IV] . 
We  need  some  kind  of  real-time  operator  interface  for  a control 
database  and  a simulation  database.  These  interfaces,  we 
believe,  will  come  in  two  "flavors.”  One  will  be  a high-tech, 
single-use  interface  - the  very  flashy  kind  of  VR  - with  fine 
graphics  and  perhaps  goggles,  gloves,  and  the  like.  But  this  is 
an  expensive  solution,  at  least  in  the  near-term,  and  will  be 
limited  to  a fairly  small  number  of  specialized,  but  some  very 
important,  areas.  We  need  to  explore  this.  So  we  will  be 
building,  at  NIST,  a testbed  where  we  can  try  out  ideas  with  this 
kind  of  interface. 

A second  type  of  interface  would  be  a multi-user,  shared- 
data  environment  interface,  where  we  will  have  many  people  in 
different  parts  of  the  world  (perhaps  a factory  manager  in  Hong 
Kong,  a production  executive  in  New  York,  a consultant  in 
Cleveland,  and  a shop  floor  supervisor  in  Baltimore) . All  of 
them  probably  have  some  low-tech  terminals,  but  they  need  to  look 
into  these  databases  together,  to  navigate  through  them,  to 
manipulate  them,  to  discuss  them,  and  to  do  things  in  real-time, 
and  probably  with  some  kind  of  natural  language  interface 
(because  we  can't  rely  on  them  being  highly  trained  in  the  use  of 
these  interfaces) . These  are  the  two  kinds  of  virtual  reality 
that  we  are  looking  at  - one  is  probably  what  we  would  call  today 
graphics -based  virtual  reality  and  the  other  is  probably  what  we 
would  call  text-based  VR. 

One  of  the  NIST  roles  will  be  to  investigate  the  best 
possible  standards  for  presenting  information  in  either  of  these 
modes.  Another  will  be  to  try  to  define  standard  interfaces  for 
communicating  information  between  databases  representing 
manufacturing  systems  and  operator  interfaces. 

So  these  are  some  of  our  thoughts  today.  They  may  not  be 
perfect  or  complete  thoughts,  but  this  workshop  begins  our  effort 
to  bring  together  people  who  can  tell  us  how  that  picture  can  be 
changed.  That  is  really  what  we  would  like  to  focus  on  today. 

The  objectives  for  the  workshop  are  to:  (1)  identify 
industry  pull  in  order  to  identify  what  we  should  be  doing;  (2) 
identify  technology  push  in  order  to  learn  what  technology  can  do 
for  us  today  and  over  the  next  5 years,  and  (3)  target  meaningful 
opportunities  in  the  overlap  of  these  pulls  and  pushes  in  order 
to  learn  where  NIST  can  make  a difference.  This  workshop  is  the 
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means  to  bring  together  experts  who  can  help  us  learn. 

From  NIST's  perspective,  there  are  three  principal  issues, 
what  we  would  like  to  have  answers  for  at  the  end  of  the  day: 

(1)  What  kinds  of  data  should  we  focus  on?  What  things 
ought  to  come  across  the  interface  for  the  shop  floor  supervisor, 
the  remote  consultant,  the  factory  manager,  and  all  of  the 
different  kinds  of  people  who  really  need  data?  What  kinds  of 
things  should  we  bring  to  the  operation  through  the  operator 
interface? 

(2)  What  about  the  human  factors  issue,  such  as  the  best 
formats  for  presenting  this  information?  There  are  ways  of 
optimizing  how  information  is  presented  to  people,  and  we  need  to 
understand,  in  the  context  of  manufacturing  data,  what  these 
formats  are. 

(3)  Finally,  we  need  to  find  the  best  technologies  to 
deliver  the  information  to  the  user.  After  we  understand  what  we 
want  to  present  and  how,  optimally,  we  want  to  present  it,  we 
need  to  look  at  the  technical  issue  of  how  it  can  be  done,  where 
do  we  have  to  drive  the  technology  in  order  to  do  it,  what 
research  is  required. 

We  will  try  to  arrive  at  a consensus  solution,  or  at  least 
how  to  proceed  toward  a solution.  To  get  underway,  I would  like 
to  introduce  Dr.  Mark  Luce,  who  will  give  us  a broader 
perspective  of  this  program. 

3.2  DR.  MARK  LUCE 

The  larger  NIST  program,  under  which  this  workshop  is  a 
part,  concerns  systems  integration  for  manufacturing 
applications.  The  NIST  program  is  supporting  the  National 
Information  Infrastructure  (Nil)  and  the  government  initiative 
for  High  Performance  Computing  and  Communications  (HPCC) . 

The  overall  focus  of  the  program  is  to  address  the  problems 
of  integrating  manufacturing  applications,  including  design, 
planning,  and  production.  The  SIMA  (Systems  Integration  for 
Manufacturing  Applications)  Program  was  established  in  1994.  The 
HPCC  application  for  advanced  manufacturing  was  then  included  in 
the  program. 

The  overall  objective  is  to  support  the  application  of  high 
performance  computing  into  various  areas,  including:  education 
for  life-long  learning,  energy  management,  advanced 
manufacturing,  and  others.  Some  of  the  research  areas  are 
addressed  by  other  government  laboratories.  NIST  is  addressing: 
(1)  concurrent  engineering;  (2)  systems  integration;  (3)  protocol 
for  electronic  data  exchange;  and  (4)  virtual  design 
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technologies.  The  NIST  role  is  to  support  the  development  of 
standards,  with  an  emphasis  on  the  application  of  information 
exchange.  The  NIST  role  is  to: 

* Produce  new  methodologies  for  developing  standards  for 
integrating  systems  using  HPCC  technology. 

* Develop,  demonstrate,  and  recommend  prototype  standards. 

* Lead  national  and  international  standards  experts. 

* Produce  testing  methods  and  services  for  information  exchange 
standards . 

* Perform  basic  and  applied  research. 

The  SIMA  Program  goal  is  to: 

* Provide,  in  five  years  (by  means  of  a collaborative  work 
environment)  an  integrated  strategy,  methods,  and  standards  for 
integrating  a set  of  commercial  software  applications  which 
support  manufacturing. 

* Target  such  applications  as  design,  planning,  production 
control,  and  simulation. 

* Target  industry  domains,  with  an  emphasis  on  the  discrete 
parts  and  mechanical  parts  manufacturing  industry,  process 
plants,  and  apparel  manufacturing. 

SIMA  Program  environments  include:  the  Manufacturing 
Systems  Environment  (MSE) , which  concerns  the  research  and 
development  of  integration  strategies,  methods,  and  systems;  the 
Standards  Development  Environment  (SDE) , which  concerns  the 
development  of  tools  and  methodologies  for  information  exchange 
standards;  and  the  AMSANT  (Advanced  Manufacturing  Systems  and 
Networking  Testbed)  Environment,  which  concerns  the  development 
of  an  internal  testbed  and  facilitating  access  to  the  information 
highway. 

Within  the  SIMA  Program  environment,  the  objective  of  the 
MSE  is  to: 

* Develop,  test,  and  implement  solutions  to  achieve  a 
manufacturing  data  exchange  across  engineering  applications. 

* Design,  plan,  and  produce. 

* Develop  information  models  for  manufacturing  processes  and 
resources . 

* Develop  interface  specifications  between  applications. 

* Define  system  the  architecture  for  the  data  exchange  of 
products  and  processes. 
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The  emphasis  on  MSB  applications  is  to  determine  how 
different  applications  share  and  exchange  information,  and  how 
information  is  represented.  The  objective  of  the  SDE  is  to 
develop  and  assist  industry  with  the  implementation  of  an 
information  exchange  standard.  The  objective  of  the  AMSANT  is  to 
develop  a NIST-wide  testbed.  Part  of  the  testbed  will  be  a VR 
room.  We  hope  to  get  the  testbed  implemented  by  1 October  1994. 

in  FY  94  we  have  been  scoping  the  problems.  In  FY  95  we 
will  define  solutions,  in  which  this  workshop  is  part  of  the 
process.  In  FY  96-97  we  will  develop  new  integration 
capabilities,  and  in  FY  98  we  will  integrate,  test,  and  refine 
the  system. 

There  will  be  a number  of  workshops  dealing  with  a wide 
range  of  issues.  Contact  me,  Mark  Luce,  to  be  included  on  the 
invitation  list. 

[The  question  (Q)  and  answer  (A)  session  for  Dr.  Luce 
included  the  following] . 

Q:  Is  one  of  your  goals  to  develop  a reference  model  document 

for  the  standards? 

A:  Yes. 

Q:  Will  industry  and  academia  have  access  to  testbed? 

A:  Yes.  We  expect  to  use  it  as  a place  for  collaboration,  such 

as  with  CRADAS  (Cooperative  Research  and  Development 
Agreements) . 

Q:  Can  multiple  laboratories  collaborate  over  networks  with 

NIST? 

A:  Yes. 

3.3  DR.  ERNIE  KENT 

Thank  you  Dr.  Luce. 

We  would  like  the  users  to  let  us  know  what  we  should  have 
in  our  VR  testbed.  We  can  have  whatever  network  people  want, 
including  lower  speed  network  teleconferencing;  remote  operations 
will  be  included  in  the  testbed.  As  for  the  construction  of  the 
VR  testbed  by  Oct  1994:  the  walls  may  be  in  place,  but  not  the 
equipment. 


3.4  DR.  BOWEN  LOFTIN 

My  purpose  here  is  to  walk  you  through  a survey  of  the 
technology  of  Virtual  Reality  [please  see  the  Charts  in  Appendix 
V]. 
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VR  is  known  by  many  names  (chart  2) ; all  mean  the  same 
thing,  or  different  things,  depending  on  who  is  doing  the 
defining.  My  definition  is  a very  broad  one  (chart  3) . If  we 
can  make  an  individual  or  group  believe  they  are  somewhere  else, 
then,  as  far  as  I am  concerned,  that  is  virtual  reality.  A video 
game  can  be  virtual  reality  because  one  can  become  fairly 
immersed  in  that  environment.  A good  novel  can  even  be  virtual 
reality  in  that  sense.  This  is  a very  broad  definition  because  I 
do  not  think  we  should  exclude  particular  features  or  flavors  of 
technologies  that  can  provide  this  kind  of  immersive  experience 
because  they  all  can  be  useful  to  us,  depending  on  the 
application. 

I have  a list  (chart  4)  of  what  I think  are  important 
applications,  but  it  is  not  a complete  list.  The  human/machine 
interaction  is  a key  application.  For  thousands  of  years  we  have 
built  machines  and  we  have  adapted  ourselves  to  our  machines  - 
maybe  we  are  finally  at  the  point  where  we  can  have  a machine 
adapt  to  us.  A computer  keyboard  is  not  a natural  interface.  We 
have  to  force  ourselves  to  adapt  to  our  machines.  The 
empowerment  of  disabled  humans  is  another  very  important 
application.  Or  to  empower  researchers  by  "enlarging”  or 
"shrinking”  objects  or  environments  of  interest  (such  as  allowing 
researchers  to  move  around  molecules) . The  underlined 
applications  (in  chart  4)  are  those  I work  in,  with  my  largest 
effort  in  training  and  education  applications. 

Beginning  the  formal  overview  (chart  5) : these 
technologies,  when  brought  to  bear  in  some  combination,  will 
provide  what  we  call  virtual  reality.  Not  all  are  required  - a 
couple  may  be  enough. 

Many  VR  applications  (chart  7)  can  be  accomplished  on 
desktop  PC's,  such  as  Amigas.  I tend  to  work  in  the  middle  of 
the  computer  spectrum,  on  workstations.  In  terms  of  the  key 
elements,  those  are  lots  of  problems  in  producing  good  graphics. 
Parallel  processing  is  a growing  method.  The  bottom  line  is 
speed  and  cost,  and  the  two  are  interrelated.  High  speed  usually 
implies  high  cost. 

The  heart  of  a graphics  application  (chart  8)  has  to  be  a 
way  to  render  it  - the  graphics  software.  Normally,  the 
environments  consist  of  polygons.  To  make  a curved  surface,  we 
need  lots  of  polygons.  The  polygon  count  is  usually  the  measure 
of  how  fast  you  can  render  a particular  scene.  The  best 
workstations  today  can  usually  render  tens  of  thousands  of 
polygons  in  real  time  (about  30  frames/ second) . We  commonly  work 
with  40-60  thousand  polygons  systems  and  render  them  at  rates 
around  25  frames /second.  But  filling  in  the  polygons  with 
colored  pixels  also  requires  effort,  time,  and  hardware.  Texture 
mapping  is  something  which  has  been  used  widely  and  makes 
attractive  environments,  but  texture  maps  are  just  pixels  which 
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are  turned  off  and  on;  you  have  no  way  of  using  these  pencils  for 
object  collision  detection.  So  polygon  count  is  the  way  we 
determine  interactions.  High  resolution  interactions  (to 
determine  where  two  objects  interact)  require  many  polygons. 

There  are  many  books  written  about  ray  tracing.  Realistic 
environments  require  ray  tracing  and  lighting  models. 

There  are  many  machines  using  multiple  processing;  but  we  do 
not  know  yet  how  to  use  them  appropriately. 

One  of  our  goals  is  to  develop  accessible  tools  to  enable 
almost  anyone  to  build  and  alter  complete  virtual  environments. 

We  shouldn't  be  held  captive  by  a few  people  who  know  how  to 
build  VR  environments  because  this  will  lead  to  high  cost  and 
frustration.  The  tools  will  allow  VR  applications  to  become 
widespread. 

The  two  basic  divisions  in  displays  (chart  9)  are  the  non- 
head  mounted  vs.  head-mounted  approaches.  The  military  has  been 
using  projected  dome  systems  for  years.  They  are  common  in 
military  training  environments,  but  they  are  extraordinarily 
expensive.  They  require  big  facilities  and  high  performance 
systems  costing  millions  of  dollars  But  the  day  of  the  big 
projection  dome  is  about  gone.  There  is  a limited  market.  They 
are  dinosaurs.  The  cost  of  cave  technology,  however,  is  coming 
down  rapidly. 

I tend  to  work  with  head-mounted  displays  because  we  are 
interested  in  providing  people  with  good  mobility  in  virtual 
environments,  and  the  focus  is  on  individual  use,  as  opposed  to 
group  use.  (In  a cave  with  five  people,  one  person  controls  the 
viewpoint  for  the  other  four  people) . 

There  are  two  major  types  of  head-mounted  displays.  One 
type  is  based  on  CRT  technology,  and  another  type  is  based  on  LCD 
technology.  There  are  several  CRT-type  displays.  Direct 
displays  are  fairly  common.  There  are  some  very  expensive  piped 
displays  consisting  of  fiber  optics  bundles  (costing  as  much  as 
$3  million  for  a helmet) . Direct  displays  will  improve  even  as 
their  cost  decreases. 

The  LCD  displays  are  popular  because  they're  lighter  than 
CRT  displays  and  have  lower  power  requirements  (less  volts  on 
your  head) . Ongoing  ARPA  projects  can  lead  to  higher  resolution, 
lower  cost  LCD  displays.  Within  a year  or  two  there  will  be  very 
good  LCD  systems  on  the  market  at  fairly  low  cost.  The  retinal 
display  (a  developmental  technique)  uses  a low  power  base  to 
paint  the  image  directly  on  the  retina,  just  like  an  electron 
beam  on  a CRT.  It  provides  a wide  field  of  view  and  high 
resolution. 
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The  issues  are  resolution  and  field  of  view  (FOV) . You  want 
both,  but  may  get  neither.  Resolution  has  always  been  a problem, 
especially  in  the  LCD  system.  Only  in  the  past  year  or  two  has 
there  been  significant  improvement.  Generally,  you  trade-off 
resolution  for  field  of  view,  or  vice  versa. 

About  a 60-degree  field  of  view  is  minimum  for  immersion  in 
the  environment.  In  the  future  we  should  be  able  to  get  high  FOV 
and  high  resolution  at  reasonable  cost. 

Q:  Has  anyone  tried  to  do  f ovial-peripheral  kinds  of  things, 

with  variable  resolution? 

A:  Yes.  Early  in  the  dome-based  approach,  there  was  high 

resolution  in  the  direction  in  which  you  were  looking  and 
lower  resolution  around  that.  It  is  complicated 
computationally,  however.  This  has  been  pretty  well  solved 
in  dome  systems,  but  it  has  not  been  well-addressed  in 
helmet -mounted  systems.  It  is  a very  good  approach  because 
once  you  move  a few  degrees  outside  your  fovial  view,  the 
resolution  required  is  low.  The  blending  of  the  field  is 
difficult. 

Q:  What  about  eye-tracking? 

A:  Some  approaches  have  used  eye-tracking,  but  it  is  still  a 

tough  problem.  We  haven't  been  able  to  build  a system  with 
good  eye-tracking  and  good  displays.  There  is  a whole  issue 
of  eye-tracking  related  to  helmets.. 

Q:  What  about  boom  systems? 

A:  I haven't  mentioned  them,  but  they  have  been  around  for 

awhile.  The  boom  system  can  offer  fairly  high  resolution 
and  field  of  view.  It  is  usually  a CRT-based  system.  The 

head  doesn't  get  tired  because  it  doesn't  have  to  support 

the  weight  of  the  system.  There  is  a company  offering  a 
coinmercial  system  able  to  support  about  20  pounds.  They 
cost  about  $15 OK  and  up,  so  they  are  not  very  cheap. 

Q:  Can  you  discuss  VR  sickness? 

A:  This  is  a topic  in  itself  which  I won't  address  in  detail. 

There  are  all  kinds  of  problems,  but  we  think  they  fall  in  a 
few  categories.  Nausea  tends  to  come  from  conflicting 
sensory  cues,  primarily  when  the  vestibular  and  visual  cues 
don't  match.  If  you  don't  have  the  correct  interocular 
distance  in  your  display,  your  eyes  will  adapt.  But  when 
you  remove  the  system,  you  will  have  ocular  motor  problems, 
eye  strain,  headache,  etc.  You  need  to  transition  the  user 
back  into  the  world  - before  they  drive  a car  or  fly  a 
plane,  for  example,  to  avoid  accidents.  Someday,  someone  is 
going  to  get  sued  after  a car  wreck.  In  car  applications, 
nausea  is  rare,  but  ocular  motor  problems  are  common. 

Most  of  our  VR  sessions  run  45  minutes  to  an  hour.  In  a 


Application  of  Virtual  Reality  to  Manufacturing  Workshop 


13 


compelling  VR  environment,  people  may  stay  longer  than  they 
should  and  can  suffer  eye  strain  or  other  adverse  affects.  In 
our  lab  a training  session  might  last  25  minutes. 

All  of  our  ears  are  different,  so  to  provide  good  3-D 
localization,  you  need  a model  of  each  individual's  ear  (chart 
10) . The  main  problem  is  not  localization  in  azimuth,  but  in 
elevation,  which  depends  more  on  the  shape  of  the  ear. 

There  are  various  technologies  to  provide  the  user  with  a 
sense  of  touch  (chart  11) . The  electrorheological  glove  is  an 
experimental  system  involving  a fluid  enclosed  in  a double  layer 
glove  which  can  change  viscosity  based  on  the  application  of 
electrical  fields,  becoming  stiff  or  soft.  The  available 
technology  is  interesting,  but  crude. 

There  is  only  one  company  building  a thermal  system.  This 
provides  a very  high  fidelity  temperature  change.  For  example, 
you  can  grasp  a virtual  glass  of  iced  tea  and  it  will  feel  cold 
to  the  touch.  But  we  need  to  integrate  these  systems  to  allow 
two  or  three  properties  at  one  time,  such  as  tactile,  force,  and 
thermal. 

Olfactory  displays  (chart  12)  have  been  around  for  awhile, 
and  they  can  be  important  in  some  applications.  For  virtual 
surgery,  the  smell  of  the  opened  human  body  can  be  important  to 
the  surgeon.  In  other  applications  it  might  be  important  to 
smell  hot  oil  or  insulation  burning,  for  example.  There  are  no 
real  examples  of  taste  displays,  but  there  are  some  interesting 
possibilities. 

Eventually,  there  may  be  some  way  to  use  our  brains  directly 
as  displays.  This  is  the  ultimate  computer  - between  our  ears  - 
if  we  could  stimulate  high  resolution  at  will,  we  will  have  an 
excellent  approach  to  VR. 

Tracking  devices  (chart  13)  are  necessary,  but  very 
frustrating  things  to  work  with.  You  have  to  head-track  to  be 
able  to  follow  changing  scenery,  to  move  your  head  around.  You 
have  to  be  able  to  track  hands  and  other  body  parts  if  you  want 
to  replicate  motion.  This  is  accomplished  today  mostly  using 
magnetic  tracking  devices.  Magnetic  tracking  has  limited  range 
and  resolution.  Acoustic  trackers  also  have  limited  range,  as 
well  as  interference  problems.  Optical  trackers  are  not  bad,  but 
they  are  not  easy  to  emplace  without  permanence  - they  represent 
a big  investment  in  the  workplace.  There's  lots  of  work  needed 
to  improve  tracking  technology  for  the  manufacturing  community, 
as  well  as  other  applications. 

Input  devices  (chart  14)  have  been  used  for  many  years.  The 
VPL  Dataglove  originated  in  work  done  at  the  NASA  Ames  Research 
Center.  The  Cyberglove  is  more  precise  and  robust  than  the 
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Dataglove.  Video  capture  has  worked  very  well  for  a lumber  of 
applications.  Eye  tracking  is  still  difficult.  Voice  input  is  a 
given  - people  are  used  to  talking  to  communicate.  Myoelectric 
involves  using  the  electrical  impulses  generated  by  muscle 
action,  but  there  is  much  work  to  be  done.  The  3-D  mouse  is  not 
nearly  as  useful  as  the  2-D  mouse.  We  cannot  simply  extrapolate 
a 2-D  interface  to  the  3-D  environment.  Computer  scientists  and 
human  factors  researchers  must  work  together  to  develop  suitable 
interfaces  for  3-D  applications.  We  should  not  have  to  adapt  to 
the  machine  - the  machine  should  adapt  to  us. 

There  is  lots  of  work  going  on  in  shared  work  environments 
(chart  15) . There  are  2 predominant  ways  of  doing  shared  work 
environments  today.  In  the  first  way,  everyone  has  their  local 
database  and  graphics  generator  ability.  Then  all  I need  do  is 
send  you  data  as  I change  the  state  of  my  world.  This  is  how 
we've  done  it  for  years  with  SIMNET.  This  is  a low  bandwidth 
technique  - about  25  kbits/sec  is  usually  sufficient.  But 
everyone  needs  similar  hardware  and  systems  in  order  to 
participate.  In  the  second  way,  graphics  are  produced  at  one 
site  and  transmitted  to  other  sites.  Therefore,  multiple  sites 
can  have  low-cost  equipment.  But  there  is  a high  bandwidth 
requirement.  This  is  technique  is  the  ultimate  solution. 

We  have,  in  our  lab,  an  equipment-rich  environment  by  most 
standards  today  (chart  16) . These  are  the  objectives  of  our  lab 
(chart  17) . 

Here  are  the  research  issues  that  I believe  stand  before  us 
(chart  18).  I have  mentioned  a few  previously.  We  need  to  prove 
that  training  is  actually  accomplished  - or  doable  - using  VR 
technology.  Our  first  effort  to  do  this  on  a large  scale  was  the 
Hubble  Telescope  mission  last  year  where  we  trained  a hundred 
people  using  virtual  environments.  Everyone  who  has  trained  felt 
that  training  was  very  useful.  This  conclusion  is  not  the  result 
of  a scientific  study,  but  it  is  an  indication  of  the  usefulness 
of  the  technology  for  training  purposes.  But  the  integration 
issues  are  high.  The  registration  issue  is  knowing,  in  augmented 
reality,  where  to  put  the  graphics  representing  real  objects  (and 
their  actual  locations)  in  the  environment.  It  is  doable,  if  you 
can  control  your  environment  and  place  appropriate  sensors  in  the 
environment  (for  example,  superimposing  graphics  onto  pegboard  to 
show  where  to  string  a wiring  harness) . 

Among  many  applications  for  VR  (charts  19-33),  virtual 
reality  can  be  used  to  navigate  and  understand  many  large 
databases,  which  is  significant  if  we  are  to  make  sense  of  a 
terrabyte/hour  of  data  about  the  state  of  the  earth.  We  are 
working  with  people  in  biology  and  biochemistry  to  look  at 
virtual  representations  of  data  from  electron  microscopy  and  x- 
ray  processes  showing  the  structure  of  proteins,  viruses  and 
cells.  We  are  working  with  chemists  looking  at  molecular  docking 


Application  of  Virtual  Reality  to  Manufacturing  Workshop 


15 


mechanisms  for  designer  drug  development.  The  area  of  navigation 
and  exploration  for  large  digital  libraries  is  very  essential, 
but  not  much  as  been  published  because  some  of  the  best  and 
brightest  doing  VR  work  have  been  going  into  the  financial 
industry,  never  to  be  heard  from  again.  Large  institutions  that 
make  money  by  moving  money  around  have  learned  that  there  may  be 
ways  of  looking  at  their  massive  data  sets  using  virtual 
environments  - there  is  lots  of  work  going  on,  but  not  much  has 
been  published. 

Multisensory  perception  or  immersion  for  discovery  is  my 
personal  favorite  (chart  21) . The  human  bandwidth  is  one  that  we 
have  to  worry  about.  Most  of  us  think  in  terms  of  vision  only. 
But  think  about  how  many  variables  you  can  perceive  visually  at 
one  time.  Research  suggest  maybe  7-9  variables,  using  color  and 
everything  else  you  can  think  of.  If  I can  add  two  or  three  more 
variables  using  auditory  techniques,  and  two  more  variables  with 
optic  techniques,  etc.,  I might  be  able  to  almost  double  my 
bandwidth.  Certain  patterns,  such  as  temporal  patterns,  may  be 
detected  better  auditorally,  than  visually.  There  is  much  work 
to  be  done  to  explore  multisensory  perception  and  how  to  optimize 
the  performance  of  our  senses  to  perceive  problems  in  data. 

Q:  What  are  new  directions  for  presenting  data  that  is  now  in 

the  form  of  graphics  and  charts  (the  kind  of  data  used  in 
manufacturing  as  well  as  the  financial  community)? 

A:  I'm  convinced  that  we  can  find  ways  to  display  data  non- 

visually.  The  trick  is  to  find  what  works.  All  of  our 
lives  we  work  with  visual  data,  ignoring  other 
possibilities.  We  can  talk  with  artists  who  use  sound  or 
music.  We  need  to  find  out  the  proper  modality  between  the 
data  and  the  way  the  human  being  wants  to  model  it,  and  find 
the  right  transduction  between  the  data  and  the  human.  We 
need  to  systematically  explore  what  is  possible  in  using 
senses  other  than  visual  for  the  perception  of  data.  There 
are  bits  and  pieces  of  programs  in  data  manipulation,  but  no 
major  programs  out  of  ARPA  or  other  government  agencies. 
Nobody  has  seriously  tackled  this  problem  in  a systematic 
way  yet.  NIST  may  be  the  place  to  base  this  kind  of 
research. 

[Dr.  Lofton  then  showed  a video  illustrating  the  VR  work  at 
his  laboratory] . 

Q:  Won't  auditory  systems  will  have  a problem  in  the 

manufacturing  environment,  which  is  noisy?  Even  the  postal 
service  has  a high  percentage  of  hearing  impaired. 
Assumptions  in  the  research  would  might  not  be  relevant  in 
the  industrial  environment. 

A:  Headsets  can  isolate  the  worker  from  the  noisy 

environment.  For  hearing  impaired  people,  we  could 
use  visual  or  haptic  interfaces.  We  would  use  other 
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sensory  modalities  to  supplement  what  a person  might  be 
deprived  of  in  a given  situation. 

Q:  Data  representation  seems  to  be  drowning  in  polygons.  Is 

there  any  work  to  use  true  solid  representations  in  VR? 

A:  There's  technology  and  products  in  the  CAD  world  that  uses 

true  mathematical  solid  representation.  There  is  other  work 
to  take  real  images  and  place  them  directly  into  the  VR 
environment.  Standardization  in  this  area  is  very  poor. 

Q:  How  would  you  characterize  the  financial  size  of  the  VR 

industry  currently? 

A:  There  are  two  kinds  of  VR  companies.  One  kind  has  VR  as  one 

product  among  many.  The  other  kind  is  totally  specialized 
in  VR,  but  these  companies  are  not  generally  long-lived. 

[Audience  comment] : One  number  is  that  VR  industry 
revenues,  in  1994,  are  about  $250  million.  Caveats  for  this 
estimate  are  that:  (1)  this  represents  equipment  packaged  solely 
for  VR  and  not  shared  across  other  applications;  (2)  it  does  not 
include  location-based  entertainment;  (3)  it  does  not  include 
government  expenditures.  So  everything  else  is  $250  million, 
give  or  take  $100  million.  And  this  is  world-wide.  The 
Europeans  and  Japanese  are  investing  in  VR,  so  the  majority  of 
this  revenue  probably  is  not  in  the  U.S. 

3.5  DR.  ERNIE  KENT 

Our  next  researcher  is  from  ATR,  Inc. , which  has  been 
working  on  simulation  and  control  for  the  Post  Office,  which 
bears  a striking  resemblance  to  the  discrete  parts  manufacturing 
industry.  Clyde  Findley  is  the  manager  of  the  Control  Simulation 
division  of  ATR  (Advanced  Technology  and  Research  Corporation) . 

He  is  a systems  engineer  with  over  15  years  of  experience  in  the 
design  and  development  of  control  systems  control  languages,  and 
video  graphics  applications. 

3.6  DR.  CLYDE  FINDLEY 

A new  mail  processing  center  for  the  Sacramento,  California 
area  was  completed  in  September  1992  (please  see  the  accompanying 
charts  and  a related  paper  in  Appendix  VI) . ATR  was  asked  to  use 
its  prototype  software  called  the  Control  and  Simulation  Analysis 
Tool  (CSAT)  to  analyze  the  West  Sacramento  tray  transport  and 
staging  system.  This  would  help  to  identify  key  problem  areas 
and  develop  ways  to  modify  the  system  to  enhance  its  performance. 
The  goal  of  our  analysis  was  to  examine  several  possible  ways 
that  the  tray  transport  and  staging  system  could  be  improved  and 
to  validate  the  accuracy  of  the  CSAT  software  against  realistic 
performance  data  wherever  possible. 

The  CSAT  is  a prototype  software  system  that  was  designed  to 
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support  the  development  of  real-time  control  software  in  a 
simulated  environment.  It  is  based  on  the  NIST  Real-time  Control 
System  (RCS)  methodology.  The  RCS  employs  a task  decomposition 
technique  to  create  a hierarchy  of  well-defined  software  modules 
such  that  each  performs  small  sets  of  tasks.  These  modules  are 
designed  so  that  the  lowest  modules  correspond  exactly  to  the 
actual  hardware  devices  that  are  being  simulated.  If  these 
devices  are  physically  connected  to  the  CSAT  computer,  they  can 
be  controlled  directly,  thereby  enabling  an  operator  to  use  the 
tool  not  only  for  simulation  and  analysis,  but  for  direct 
management  and  control  of  an  actual  facility. 

While  the  CSAT  control  and  simulation  modules  are  executing, 
a parallel  user  interface  module  monitors  the  state  of  the 
simulated  environment  and  extracts  the  object  and  physical  data 
in  order  to  display  an  interactive,  3 -dimensional  view  of  the 
environment.  The  user  is  able  to  watch  and,  to  a limited  extent, 
interact  with  the  control  and  simulation  systems  while  they  are 
running. 

What  we  need  now  are  standard  interface  protocols  which  will 
allow  user  interface  software  to  communicate  effectively  with 
real-time  control  systems  and  device-level  simulation  systems. 

New  developments  on  both  sides  of  the  interface,  including 
virtual  reality  techniques  and  advances  in  human  factors 
engineering,  will  help  drive  the  requirements  for  these  protocols 
and  enable  the  development  and  use  of  generic  tools  to  support 
the  design  of  more  sophisticated  systems. 

3.7  DR.  ERNIE  KENT 

The  next  presentation  is  by  Dr.  James  Geiwitz.  He  has  a 
Ph.D.  in  cognitive  psychology  from  the  University  of  Michigan. 

He  taught  at  Stanford  and  the  University  of  California  at  Santa 
Barbara.  He  is  currently  a Principal  Research  Scientist  at  the 
Carnegie  Mellon  Research  Institute,  and  he  is  President  of 
Advanced  Scientific  Concepts.  He  is  the  author  of  13  books  on 
psychology  and  human  factors.  His  current  research  is  on 
knowledge  acquisition  technologies  and  meta-cognitive  skills  for 
problem  solving. 

3.8  DR.  JIM  GEIWITZ 

Dr.  Geiwitz 's  charts  are  given  in  Appendix  VII.  In 
addition,  post  workshop  comments  by  Dr.  Geiwitz  can  be  found  in 

Appendix  VIII. 

I would  like  to  list  some  of  the  human  factors  issues  - some 
of  which  have  been  raised  and  discussed  in  preparation  for  the 
breakout  sessions  this  afternoon.  The  chief  meta-cognitive  skill 
is  to  define  a problem  in  some  representation  that  affords  a 
solution. 
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The  first  charts  show  a restatement  of  the  workshop's 
objectives  so  that  I can  orient  myself  to  what  we  are  doing  here. 

We  have  a standardization  challenge  here.  One  of  the  issues 
I think  is  the  role  of  reference  models  in  this  kind  of 
standardization.  We  are  also  looking  at  information  technology. 
How  do  you  get  the  advice  of  the  expert  into  the  system  you  are 
building?  The  virtual  reality  is  the  focus  of  the  workshop,  but 
it  also  has  a distributed  objective.  We  want  to  look  at  shared 
synthetic  environments  and  cooperative  learning  and  work  in  that 
environment.  The  participants  may  be  in  different  locations 
across  the  country. 

In  this  workshop  we  are  really  talking  about  the  design  of  a 
product  consisting  of  some  kind  of  virtual  reality  workstation 
that  would  be  useful  in  conjunction  with  some  kind  of 
manufacturing  system.  The  vision  is  to  integrate  the  three  areas 
in  which  we  are  working  in. 

As  a psychologist,  my  view  of  VR  is  that  it  is  an 
hallucination.  Virtual  reality  exists  in  a continuum  of 
definitions  along  some  direction,  which  we  will  talk  about.  But 
a good  working  definition  is:  occupying  an  environment  other 
than  the  physical  environment  of  the  moment.  One  dimension  for 
VR  might  be  the  ambitious  vs.  not  ambitious  ends  of  the  VR 
spectrum.  The  dimension  might  have  something  to  do  with  the 
degree  of  similarity  of  the  virtual  world  to  the  real  world.  The 
number  of  modes  (such  as  sensory)  also  has  something  to  do  with 
the  level  of  ambition  of  VR. 

A definition,  which  I think  is  from  NASA  Ames,  is:  "Virtual 
Reality  is  the  true-to-form  representation  of  objects  in  the 
real-world  and  their  interactions."  Television  and  telerobotics, 
by  their  definition,  is  a presentation  of  the  real  world,  not  a 
representation  of  the  real  world.  And  artificial  worlds  are  not 
necessarily  constrained  by  real  world  limitations  (such  as  the 
laws  of  physics) . 

Comment  (from  audience) : Text-based  VR  may  be  a major 
practical  form  of  VR  for  machine  shops. 

The  primary  human  factor  issue  is:  what  does  the  user  need 
to  know;  when  does  he  need  to  know  it;  and  in  what  form  is  it 
most  conducive  for  effective  performance.  Who  needs  virtual 
reality?  What  is  it  good  for?  Some  applications  can  get  by  with 
something  less  ambitious  and  considerably  less  costly. 

Comment  (from  audience) : One  dimension  of  VR  might  be  the 
degree  to  which  the  user  is  required  to  use  his  imagination  (as 
in  reading  a novel  versus  looking  at  graphics) . The  use  of  the 
imagination  might  be  stimulating  to  some,  but  it  might  detract  or 
distract  from  the  task  at  hand  (e.g.  manufacturing  a part 
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effectively  and  efficiently) . 

An  example  of  an  unusual  form  of  VR  is  an  acoustic  VR  system 
being  developed  by  Dr.  Roberta  Klatzky  (of  Carnegie  Mellon 
University  and  who  is  the  wife  of  Dr.  Geiwitz)  and  Dr.  Reginald 
Golledge  (University  of  California  at  Santa  Barbara) . It  is  a 
prototype  GPS-based  navigation  system  for  the  blind  which 
announces  the  location  of  objects  through  stereo  headphones 
("bench  here,  bench  here") . The  system,  connected  to  a computer 
in  a backpack,  creates  a virtual  landscape  for  the  blind  user. 

We  offer  a definition  of  manufacturing  systems,  along  with  a 
diagram  of  manufacturing  as  a feedback  system  (in  Appendix  VII) . 
The  human  factors  issues  depend  on  the  task.  As  we  said,  the 
basic  human  factors  issues  are  (1)  What  does  the  user  need  to 
know  to  perform  the  task;  (2)  when  does  he  need  the  knowledge; 
and  (3)  in  what  form  is  the  knowledge  most  conducive  to  effective 
use. 


3.9  LUNCH  BREAK:  VIDEOS 

[The  lunch  break  included  the  showing  of  VR-related 
videotapes.  Sandia  videotapes  included:  Visualization  Facility 
Walk  Through j VR/ Hypermedia  for  Training  (demonstrating  full  body 
representation) ; Enhanced  Telepresence  Interactive  Modeling; 
Determining  Wheelchair  Access;  and  Assembly  of  Parts 
Visualization.  A videotape  from  Boeing  showed  Virtual  Reality 
Research  at  Boeing,  where  VR  is  permitting  the  production  of 
aircraft  with  full  physical  work-ups  during  the  design  process 
(the  designer  can  walk  inside  the  VR  model  of  the  aircraft),  as 
well  as  providing  augmented  reality  for  laying  out  wiring 
harnesses.  A videotape  from  United  Technologies  Research  Center 
showed  a VR  conference  room  as  a testbed. ] 

3.10  DR.  JIM  GEIWITZ  [continued  after  lunch] 

There  have  been  few  relevant  research  studies  on  the  issues 
we're  concerned  with.  If  these  systems  do  not  work,  it's  costly. 
They  had  better  be  effective  or  they  are  not  going  to  be  used. 

The  chart  (Appendix  VII)  summarizes  a few  relevant  studies  and 
their  key  findings.  But  there  have  been  no  major  field  studies 
of  automated  manufacturing  (and  the  last  major  field  study  of  any 
kind  of  manufacturing  was  in  1935) . In  this  team  session,  we 
should  consider  the  various  target  tasks  and  the  various  human 
factors  issues  which  are  listed  in  the  chart  (Appendix  VII) . 

Knowledge  acquisition  is  important.  In  the  knowledge 
acquisition  guidebook  are  described  55  different  knowledge 
acquisition  techniques  (KAT) , along  with  the  circumstances  in 
which  they  are  identified  or  contradicted.  Psychologists 
generally  distinguish  between  semantics  and  procedural  knowledge. 
The  former  is  knowing  what  an  object  is,  and  the  latter  is 
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knowing  how  to  perform  some  task.  KATs  are  needed  because 
experts  may  not  be  able  to  articulate  procedural  knowledge. 

We  need  many  more  models  of  the  manufacturing  process  — we 
have  been  deficient  in  this  area.  The  application  of  VR  to 
training  is  much  more  admired  than  the  VR  applications  to 
manufacturing.  And  part  of  the  reason  is  that  we  don't  have  as 
good  models  for  the  manufacturing  process  that  we  have  for  the 
training  process.  In  particular,  we  need  models  of  autonomous 
interactive  agents,  such  as  are  in  SIMNET. 

3.11  DR.  HAROLD  VAN  COTT 

Like  many  new  fields  of  technology,  virtual  reality  has 
developed  in  a haphazard,  sporadic  fashion.  It  tends  to  be 
applications  oriented.  A lot  of  the  generic,  cross-cutting 
issues  that  are  fundamental  to  the  development  of  the  field  and 
its  proper  exploitation  and  commercialization  dropped  through  the 
cracks  — basic  questions  don't  get  answered.  So  two  years  ago  I 
initiated  a budget  at  the  National  Research  Council  on  virtual 
reality  in  conjunction  with  the  Committee  on  Human  Factors  and 
the  Board  on  Computer  Science  and  Telecommunications.  They  are 
looking  at  the  broad  field  of  virtual  reality  - what  the  state  of 
the  technology  is,  what  the  burning  issues  are,  and,  in 
particular,  they  are  looking  to  devise  an  agenda  for  cross 
cutting  research.  The  project  is  nearing  completion. 

The  report  (about  750  pages)  should  be  published  in  October 
1994.  The  study  was  funded  by  12  different  government  agencies, 
such  as  NASA,  ARPA,  NSA,  Coast  Guard,  and  several  others.  The 
report  will  be  available  from  the  National  Academy  Press. 

3.12  DR.  ERNIE  KENT 

The  plenary  session  will  break  into  small  groups  to  get  some 
good  one-on-one  discussions  and  interactions,  and  then  it  will 
return  for  another  plenary  session  to  discuss  the  results  of 
those  groups.  The  charge  to  the  groups  will  be  for  each  group  to 
put  together  its  vision  of  where  VR  in  manufacturing  could  go  in 
the  next  five  years.  What  things  are  possible?  What  things  must 
be  done?  What  are  the  means  we  have  to  bring  to  bear?  What  are 
the  questions  we  have  to  ask?  Take  some  guesses  and  try  to 
project  yourself  into  the  future.  In  particular,  those  in  the 
manufacturing  community  should  try  and  put  some  demands  on  us. 
Tell  us  that  this  is  the  kind  of  information  you  need.  These  are 
the  kinds  of  things  you  need  to  look  at.  I would  like  the  human 
factors  people  to  put  demands  on  us,  and  the  VR  people  to  say  ”we 
can't  do  that  for  a hundred  years.” 

The  moderator  assigned  to  each  group  will  summarize  what  the 
groups  came  up  with.  Then  we  will  have  a plenary  discussion 
where  everybody  can  voice  their  opinions  on  what  the  other  groups 
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came  up  with  and  arrive  at  a summary  conclusion  of  what  VR  for 
manufacturing  will  look  like  over  the  next  five  years. 

[Three  groups  were  formed,  each  with  a mix  of  human  factors, 
virtual  reality,  and  manufacturing  experts.  They  deliberated  and 
returned  to  the  plenary  sessions,  where  Dr.  Kent  continued.] 

Now  that  the  group  meetings  are  completed,  we  will  have 
three  summary  statements  from  the  groups. 

3.13  WORKING  GROUP  1:  BAILY  BUNCH 

One  of  the  things  we  discussed  was  that  it  was  hard  to 
describe  the  nature  of  manufacturing.  And  a cost/benefit 
analysis  would  require  a definition  of  manufacturing.  We  agreed 
that  manufacturing  encompasses  processes,  planning,  materials, 
controls,  scheduling,  resources  (people  and  equipment) , testing, 
integration  (assembly) , maintenance,  training,  and  other  things. 

We  noted  that  three  or  four  things  that  are  peculiar  to 
manufacturing  contribute  to  a goodness  in  VR.  From  a VR  point  of 
view,  you  can  break  manufacturing  into  operations  (or  processes) , 
maintenance,  and  an  aspect  of  training.  We  took  those  three 
items  and  asked  what  they  mean  from  a VR  point  of  view. 

For  operations,  there  is  a goodness  in  VR  for 
process/ factory  integration;  optimization;  situational  awareness 
(monitoring) ; safety;  real-time  processes;  and  achieving  agile 
manufacturing  and  just  in  time  (JIT)  distribution. 

Relating  to  the  maintenance  part  of  the  manufacturing 
process,  we  came  up  with  these  terms:  teleoperation,  remote, 
integration  of  maintenance  and  operations,  repair,  skills, 
safety,  and  tools.  Teleoperation  and  remote  control  of 
maintenance,  in  particular,  we  may  want  to  operate  with  VR. 

We  looked  at  training,  and  the  pervasiveness  of  training 
jumped  out  all  over  the  place.  We  listed  benefits, 
mistakes/ identification,  cost,  reconfiguration,  3D,  training 
technology,  worker  safety,  and  process  feedback. 

We  felt  it  more  important  to  talk  about  a description  of 
things  that  would  go  into  the  national  testbed  than  to  try  to 
respond  to  the  questions  directly. 

3.14  WORKING  GROUP  2:  GEIWITZ  GROUP 

We  found  that  the  whole  company  can  use  VR  in  design, 
manufacturing,  finance,  sales,  and  training.  We  need  more 
generalists  in  VR,  and  fewer  specialists.  The  integration  of 
domain  models  is  a major  problem.  By  using  VR  for  designing  and 
testing  (simulation) , we  can  prevent  errors  in  manufacturing. 
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We  decided  that  standards  and  protocols  were  needed  to 
interact  with  databases  in  VR.  Protocols  are  required  to  connect 
CAD  with  process  planning,  to  connect  process  planning  with 
simulation  and  scheduling.  Reference  models  are  needed  as  well; 
there  are  perhaps  seven  hierarchical  control  layers  between  the 
Information  Superhighway  and  the  manufacturing  process. 

We  agreed  that  graphics  are  basic  to  VR  to  visualize  a 
problem  and  for  knowledge  acquisition.  We  need  VR  in  the  tools 
and  the  various  system  development  phases.  But  we  need  to 
understand  the  domain  before  we  understand  the  application  of  VR. 
We  need  to  understand  the  manufacturing  requirements  before  we 
decide  that  VR  is  an  appropriate  technology  and  where  it  should 
be  applied.  We  struggled  with  this  in  the  breakfast  session. 

3.15  WORKING  GROUP  3:  FINDLEY'S  FLOCK 

We  decided  that  the  problem  is  all  about  information 
management  and  how  best  to  represent  the  knowledge  we  may  or  may 
not  have  about  the  manufacturing  processes.  We  need  to  know  how 
to  manage  large  databases,  and  how  to  view  information  from 
different  perspectives. 

VR  to  be  used  to  monitor  real-time  processes  is  much  more 
difficult  than  other  applications,  due,  in  large  part,  to  the 
need  for  new  sensor  technology. 

VR  must  support  changing  information  structures  and  models. 
With  standard  representations  of  requirements  and  designs,  VR  can 
help  create  alliances  of  companies  - a virtual  manufacturing 
system. 

VR  can  be  used  to  represent  conceptual  patterns  of  reality. 
It  can  be  used  effectively  to  support  design  and  what-if  analysis 
in  a simulated  world,  as  for  physics-based  models  or  task-based 
control  systems. 

3.16  DR.  ERNIE  KENT 

We  will  be  establishing  a process  of  workshops  at  NIST  on 
this  topic.  In  conclusion; 

* This  group  should; 

stay  in  contact 

share  in  information  and  insights 
establish  lines  of  communication 

* NIST  will  be  investigating  these  issues  and  providing 
facilities  so  that  others  can  investigate  these  issues  as 
well 

* NIST  is  inviting  you  to  use  our  resources 

* Eventually,  we  will  issue  recommendations  for  standards  (for 
interfaces,  human  factors,  etc.) 
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* The  ATP  program  cognizant  of  this  program 

* You  will  receive  a physical  copy  of  the  workshop  report 

* In  future  meetings,  we  want  to  bring  in  software  suppliers 

This  has  been  a long,  hard  day,  but  I think  a productive 
one.  I would  like  to  give  you  all  my  personal  thanks  for  being 
here  today.  I think  it  certainly  has  been  very  useful  for  us  and 
I hope  that  we're  all  going  to  be  able  to  remain  in  contact.  I 
think  we've  developed  some  good  associations  here,  and  I hope 
they  will  be  as  profitable  for  you  as  I believe  they  are  going  to 
be  for  us.  Thank  you  very  much  for  attending  [applause]. 
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APPENDIX  I 

INVITATION  PACKAGE 

Agenda 

Workshop  Objectives 
Breakout  Session  Topics 
Registration  Form 

Direction  & Map  To  Gaithersburg  Hilton 
Background  Information  For  The  Survey 
Enquiry  Form 
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TO: 


Designated  Respondent 


FROM:  Dr.  Joe  Iseroan 

(301 )-983-2465  - VOICE 
(301 )-983-2509  - FAX 

DATE:  27  July  1994 

Cover  + 6 Pages 

Subject:  NIST  Workshop  - APPLICATION  OF  INFORMATION  AND 

INTERFACE  TECHNOLOGY  FOR  VIRTUAL  AND  DISTRIBUTED 
MANUFACTURING  OF  DISCRETE  PARTS  - 9 August  1994 

(1)  The  Intelligent  Systems  Division  (ISD)  of  the  Manufacturing 
Engineering  Laboratory  (MEL)  at  the  National  Institute  of 
Standards  and  Technology  (NIST)  is  at  the  forefront  of  a 
commitment  in  the  area  of  standardization  of  protocols  for  the 
application  of  information  and  interface  technology  for  virtual 
and  distributed  manufacturing  of  discrete  parts.  As  part  of  this 
effort,  ISD  is  currently  developing  an  initiative  in  human 
interfaces  into  manufacturing,  emphasizing  integration  of  human 
factors,  virtual  reality,  and  manufacturing  systems  technologies. 
One  of  the  ISD  priority  items  is  a one-day  workshop  on  Tuesday,  9 
August  1994,  with  registration  starting  at  8:00  am. 

(2)  The  workshop  will  be  at  the  Gaithersburg  Hilton,  620  Perry 
Parkway,  Gaithersburg,  Maryland.  A large  scale  map  and 
directions  are  attached. 

(3)  The  purpose  of  this  workshop  will  be  to  determine  a vision 
for  human  factors  in  information  and  interfaces  needed  to 
integrate  virtual  reality  technologies  into  the  virtual  and 
distributed  manufacturing  of  discrete  parts. 

(4)  As  we  discussed  in  our  telephone  conversation  today,  as  a 
leader  in  the  fields  of  Human  Factors,  Virtual  Reality,  or 
Manufacturing,  you  are  invited  to  attend  and  participate  (or  to 
designate  a participant)  in  this  workshop.  Each  invited 
participant  will  interact  in  general  and  breakout  sessions  to 
address:  (1)  a definition  of  the  requisite  information  and 


interfaces  for  virtual  and  distributed  manufacturing 
applications,  (2)  human  factors  approaches  in  information  and 
interface  technologies,  and  (3)  leveraging  of  efforts  in  specific 
topical  areas.  Robotic  Technology  Inc.  is  working  on  this 
workshop  under  contract  to  NIST. 

(5)  You  are  provided  with  brief  background  information 
describing  human  factors  and  virtual  reality  (VR)  interfaces  for 
manufacturing.  Please  faoniliarize  yourself  with  this  material. 
You  probably  also  have  been  asked  to  fill  out  the  attached 
questionnaire  ~ if  you  have  not,  we  now  request  you  to  do  so. 
Please  answer  the  questions  using  your  expert  judgment  and 
opinion. 

(6)  If  you  have  any  questions  or  comments  on  the  questionnaire, 
or  if  you  wish  to  suggest  additional  respondents  or  participants, 
please  call:  Dr.  Joe  Iseman,  Robotic  Technology  Inc.,  (301)-983- 
2465. 

( 7 ) ACCOMMODATIONS 

For  your  convenience  a block  of  rooms  has  been  set  aside  at 
the  Gaithersburg  Hilton  for  the  night  of  Monday,  8 August  at  a 
(single  or  double)  rate  of  $75.00.  Please  contact  the  Hilton 
directly  at  (301 )-977-8900  to  reserve  a room. 

(8)  POINTS  OF  CONTACT 

Cathy  Kilmer  (NIST,  ( 301 )-975“2858,  voice  and  (301)-948- 
2067,  fax)  or  Dr.  Joe  Iseman  (Robotic  Technology,  Inc.-  (301)- 
983-2465,  voice  and  ( 301  ) -983-2509 , fax)  are  additional  points  of 
contact . 

( 9 ) ACTIONS  ITEMS 

* REGISTRATION  - Please  fax  the  registration  form  for 

your  participation  in  the  workshop,  as 
soon  as  possible  to  (301 )-983-2509 . 

* QUESTIONNAIRE  - Please  fax  only  the  completed  form  (not 

the  background  information),  as  soon  as 
possible,  to  (301 )-983-2509. 

Thank  you  so  very  muchi 

ENCLOSURES : 

Draft  Agenda 

Objectives  of  the  Workshop 
Breakout  Sessions  for  the  Workshop 
Registration  Form 

Directions  and  Map  to  Gaithersburg  Hilton 
Background  Information  (for  questionnaire) 

Enquiry  Form  - Application  of  Information  and  Interface 
Technology  for  Virtual  and  Distributed  Manufacturing  of 
Discrete  Parts 


AGENDA 
WORKSHOP  ON 

APPLICATION  OP  INFORMATION  AND  INTERFACE  TECHNOLOGY 
FOR  VIRTUAL  AND  DISTRIBUTED  MANUFACTURING  OF  DISCRETE  PARTS 

PRESENTED  BY 

INTELLIGENT  SYSTEMS  DIVISION 
MANUFACTURING  ENGINEERING  LABORATORY 
NATIONAL  INSTITUTE  OF  STANDARDS  AND  TECHNOLOGY 

GAITHERSBURG  HILTON 
GAITHERSBURG,  MARYLAND 

9 AUGUST  1994 

0800-0830  REGISTRATION  (Rockville-Potomac  Suites) 

0830-0900  WELCOME  - DR.  ERNEST  KENT,  NIST. 

0900-0915  OVERVIEW  OF  SIMA  (SYSTEM  INTEGRATION  FOR  MANUFACTURING) 
PROBLEM  - MARK  LUCE,  NIST. 

0915-1015  OVERVIEW  OF  VIRTUAL  REALITY:  A NEW  TOOL  FOR 

MANUFACTURING  - DR.  R.  BOWEN  LOFTIN,  NASA  JSC. 

1015-1030  MORNING  BREAK 

1030-1100  VIDEO:  A US  POSTAL  SERVICE  APPLICATION:  CONTROL, 

SIMULATION,  AND  ANALYSIS  OF  A VIRTUAL  MAIL  PROCESSING 
FACILITY  - CLYDE  FINDLEY,  ATR. 

1100-1200  HUMAN  FACTORS  ISSUES  IN  VR  INTERFACE  DESIGN  - DR.  JAMES 
GEIWITZ,  CARNEGIE-MELLON  UNIVERSITY. 

1200-1300  WORKING  LUNCH:  VIDEO  PRESENTATIONS:  VIRTUAL  REALITY 
DEVELOPMENTS 

STANSFIELD,  SANDIA  NATIONAL  LABS 
BAILEY,  BOEING  DEFENSE  AND  SPACE  GROUP 
EDWARDS,  UNITED  TECHNOLOGIES  RESEARCH  CENTER 

1300-1315  A FRAMEWORK  FOR  SUCCESSFUL  BREAKOUT  SESSIONS  - 
DR.  ERNEST  KENT,  NIST. 

1315-1445  BREAKOUT  SESSIONS  - GROUP  1 - GEIWITZ,  CMU 

GROUP  2 - FINDLEY,  ATR 
GROUP  3 - BAILEY,  BOEING 

1445-1500  AFTERNOON  BREAK 

1500-1600  REPORTS  BY  BREAKOUT  SESSION  CHAIRPERSONS 

1600-1700  PLENARY  DISCUSSION  DEFINING  NIST  DIRECTION,  DR.  ERNEST 
KENT,  NIST 


1700 


ADJOURNMENT 


OBJECTIVES  OF  THE  WORKSHOP 


(1)  Define  a vision  of  information  and  interfaces  for  virtual  and 
distributed  manufacturing  systems: 

* the  human  role  in  manufacturing  systems 

* human  factors  viewpoints  for  information  and  interfaces  in 
virtual  emd  distributed  interactions 

* types  of  data  which  should  be  presented  by  virtual  and 
distributed  multimedia  (textual  and  graphical) 

* technologies  available  for  virtual  and  distributed  interactive 
(visual,  auditory,  and  tactile)  interfaces  (work  station 
monitors,  head-mounted  displays,  data  gloves)  and  to  support 
Immersion  (viewpoint,  movement) 

* technology  transfer  from  current  and  developmental 
entertainment  and  military  applications  systems 

* communications  support  requirements  for  expected  local  and 
wide  area  networks. 

(2)  Explore  human  factors  for  future  information  interfaces  in 
virtual  and  distributed  manufacturing  systems: 

* pulls  or  objectives:  what  are  the  user  needs? 

* pushes  or  drivers:  what  are  current  and  expected  technologies 
from  the  developers? 

* short  and  long  term  objectives  and  drivers 

* most  effective  formats  for  manufacturing  information 

* requirements  for  natural  language  interfaces 

* requirements  for  graphical  interfaces 

* requirements  for  other  sensor  interfaces. 

(3)  Realistic  leveraging  in  specific  topical  areas: 

* value  added  (potential  payoffs  of  new  approaches  for 
information  and  interfaces  for  virtual  and  distributed 

manufacturing,  particularly  in  terms  of  a competitive  edge  in 
the  global  marketplace) 

* shortfalls 

* potential  insertion  rates  for  virtual  and  distributed 
manufacturing . 

(4)  Needed  standards  and  protocols: 

* standards  for  human  interface  technologies 

* interface  standards  between  display  technologies  and 
manufacturing  data  bases. 


BREAKOUT  SESSIONS  FOR  THE  WORKSHOP 

Topics  to  be  considered  in  each  breakout  session  include: 

HUMAN  FACTORS  APPROACHES  TO  INFORMATION  AND  INTERFACES  IN  VIRTUAL 
AND  DISTRIBUTED  MANUFACTURING  SYSTEMS 

* Types  and  levels  of  information  content 

* Human  Factors  Considerations 

- Information  detail  - bandwidth,  resolution,  and  fidelity 

- Interaction  approaches  - visual,  auditory,  tactile,  and 
haptile 

* Individual  interactions 

- The  shop  floor  - shop  floor  supervisors,  shop  floor  personnel 
(machine  operators,  inspectors,  material  handlers,  and 

personnel  Involved  with  maintenance  and  repair,  the  supply 
room,  and  the  tool  room) 

- The  designer  and  planners  - CAD-CAM  designers,  design 
engineers,  process  planning  engineers,  manufacturing 

consultants 

- The  business  and  management  personnel  - plant  managers, 
schedulers,  accountants,  management  information  systems 

personnel,  and  business  consultants 

- The  external  environment 

* Collaborative  participation 

- Across  one  company 

- Among  companies 

- The  enterprise. 

EXAMPLE  APPLICATION  7;  VIRTUAL  CORPORATIONS 

* Distributed  entities  and  integrating  functions 

* Working  together  in  a synthetic  environment 

* Teleconferencing  for  ’’meeting  at  a distance;”  telepresence  in  a 
collaborative,  immersive  environment  for  joint  activities, 

including  discussions,  interviews,  planning,  and  training. 

EXAMPLE  APPLICATION  2:  VIRTUAL  AND  DISTRIBUTED  MANUFACTURING  - 
THE  FACILITIES,  PROCESSES,  AND  PRODUCTS 

* Virtual  and  distributed  design  - facilities,  processes,  and 
products 

* Simulation:  new  views  and  perspectives  unattainable  by  current 
methods 

* Virtual  and  distributed  planning  - facilities,  processes,  and 
products . 

* Simulation  and  development 

EXAMPLE  APPLICATION  3:  VIRTUAL  AND  DISTRIBUTED  TRAINING 

* Integration  of  training  for  personnel  at  distributed  sites 
with  distributed  elements 

* Simulations  or  synthetic  environments  (cost  savings,  hazardous 
environments,  unavailable  real  items) 

* Distance  learning  and  instruction 

* Students  interacting  with  unavailable  and  out-of-size 
components  and  environments 


REGISTRATION  FORM 


for  the 
WORKSHOP  ON 

APPLICATION  OP  INFORMATION  AND  INTERFACE  TECHNOLOGY 
FOR  VIRTUAL  AND  DISTRIBUTED  MANUFACTURING  OP  DISCRETE  PARTS 

PRESENTED  BY 

INTELLIGENT  SYSTEMS  DIVISION 
MANUFACTURING  ENGINEERING  LABORATORY 
NATIONAL  INSTITUTE  OF  STANDARDS  AND  TECHNOLOGY 

GAITHERSBURG  HILTON 
GAITHERSBURG,  MARYLAND 

9 AUGUST  1994 

Please  fax  this  completed  form  as  soon  as  possible  to 

Dr.  Joe  Iseman 
Robotic  Technology  Inc. 

(301 )-983-2509  (FAX) 

(301 )-983-2465  (VOICE) 


Please  Nark  One:  I Will  [ ] I Will  Not  [ ] attend  this 

workshop  on  INFORMATION  AND  INTERFACE  TECHNOLOGY  FOR  VIRTUAL  AND 
DISTRIBUTED  MANUFACTURING 

YOUR  NAME 

Last  First 

POSITION  or  TITLE 


ORGANI ZATION/ AGENCY 


ADDRESS 


PHONE 

FAX 


E-Mail 


GAITHERSBURG 
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TO: 

Designated  Respondent 

FROM:  Dr.  Joe  Iseman 

(301)-983>2465  - Voice 
(301)->983-2509  - FAX 

DATE:  27  July  1994 


Subject  Questionnaire  On  Application  of  Information  and  Interface 
Technology  for  Virtual  and  Distributed  Manufacturing  of  Discrete 
Parts 

Cover  +13  Pages 

(1)  The  purpose  of  this  questionnaire  is  to  determine  a vision 
for  human  factors  in  information  and  interfaces  needed  to 
integrate  virtual  reality  technologies  into  the  virtual  and 
distributed  manufacturing  of  discrete  parts.  You  are  asked  to 
address:  (a)  a definition  of  the  requisite  information  and 
interfaces  for  virtual  and  distributed  manufacturing 
applications,  (b)  human  factors  approaches  in  information  and 
interface  technologies,  and  (c)  leveraging  of  efforts  in  specific 
topical  areas.  The  results  of  this  questionnaire  will  allow  the 
Intelligent  Systems  Division  (ISD)  and  the  Manufacturing 
Engineering  Laboratory  (MEL)  of  the  National  Institute  of 
Standards  and  Technology  (NIST)  to  determine  needed  standards  and 
protocols  for  information  and  interfaces.  Robotic  Technology 
Inc.  is  working  on  this  questionnaire  under  contract  to  NIST. 

(2)  You  are  provided  with  brief  background  information  describing 
potential  intersections  of  human  factors  and  virtual  reality  (VR) 
for  manufacturing.  Please  familiarize  yourself  with  this 
material  before  answering  the  questions  using  your  expert 
judgment  and  opinion. 

(3)  Please  fax  only  this  completed  form  (not  the  background 
information),  as  soon  as  possible,  to  (301 )-983-2509. 

(4)  If  you  have  any  questions  or  comments,  or  if  you  wish  to 
suggest  additional  respondents,  please  call:  Dr.  Joe  Iseman, 
Robotic  Technology  Inc.,  ( 301 ) -983-2465 . 


(5)  Thank  you  so  very  much 
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BACKGROUND  INFORMATION: 

APPLICATION  OF  INFORMATION  AND  INTERFACE  TECHNOLOGY 
FOR  VIRTUAL  AND  DISTRIBUTED  MANUFACTURING  OF  DISCRETE  PARTS 

PURPOSE 

The  National  Institute  of  Standards  and  Technology  (NIST)  is 
supporting  new  initiatives  in  manufacturing  technology  which 
promise  to  have  a major  impact  on  the  national  economy.  The 
Intelligent  Systems  Division  (ISD)  of  NIST  is  exploring  the 
potential  applications  of  human  information  and  interface 
technology  for  virtual  and  distributed  manufacturing.  The 
purpose  of  the  enclosed  enquiry  is  to  determine  some  of  the 
issues  in  human  factors  wd  virtual  reality  (VR)  for 
manufacturing  from  the  user's  perspective  and  to  build  a new 
vision  for  manufacturing  systems. 

Key  questions  relate  to  the  attributes  of  human  information 
interactions  through  10  interfaces  and  their  impact  on  potential 
leveraging  of  manufacturing  technology.  Who  needs  to  look  at 
information?  What  kinds  of  information  are  appropriate  for  each 
user  to  support  real-time,  collaborative  decision-making?  What 
differences  are  there  in  information  needed  for  different  users  - 
individually  or  in  new  collaborative  process  within  a "virtual" 
presentation  environment?  What  are  the  important  issues  in  how 
users  choose  to  view  information  - both  text  or  graphics?  What 
human  factors  issues  and  design  considerations  should  be  included 
in  a VR  computer-interface?  How  can  the  information  content  and 
presentation  best  be  approached  with  VR  computer- interface 
technologies? 

NEW  INITIATIVES 

Manufacturing  technology  is  currently  a focus  of  national 
initiatives  sponsored  by  numerous  government  organizations  and 
industrial  consortia.  There  is  a commitment  of  funding  by 
prograuns  in  the  Department  of  Commerce  (Advanced  Technology 
Program  [ATP]),  Department  of  Defense  (Technology  Reinvestment 
Program  [TRP]),  as  well  as  the  Department  of  Energy,  NASA,  and 
other  Federal  and  state  agencies.  All  of  these  programs  address 
the  necessity  of  boosting  US  manufacturing  capabilities. 

In  this  regard  NIST  has  been  given  an  expanded  role  in 
standardizing  technological  infrastructures  and  applications  in 
such  programs  as: 

* Manufacturing  Research  Technology  Centers  (METCs),  which  have 
become  major  regional  technology  resources 

* The  Automated  Manufacturing  Research  Facility  (AMRF),  an  open 
architecture  test-bed  for  the  automated  manufacture  of  discrete 
parts 

* The  PDES-STEP  (Product  Data  Exchange  using  STEP  - Standard  for 
the  Exchange  of  Product  Model  Data)  Prograon  for  standardizing 
manufacturing  data  representations. 
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Virtual  Reality 

Virtual  reality  is  rapidly  becoming  technically  and 
economically  feasible.  VR  has  many  definitions  and  formats.  At 
the  ambitious  end  of  the  spectrum,  VR  allows  a user  to  have  the 
illusion  that  he  is  immersed  in  a synthetic  or  remote 
environment.  He  experiences  sensory  information,  such  as  three- 
dimensional  vision  and  binaural  sound,  that  is  generated  by  a 
computer  or  originates  from  remote  sensors  (or  is  a combination 
of  the  two  sources).  He  interacts  kinesthetically  with  computer 
generated  or  teleoperated  objects.  At  the  less  aunbitious  end  of 
the  spectrum,  VR  allows  a user  to  operate  in  a text-based 
interactive  modes  (i.e.,  on-line  dungeons  and  dragons)  or  a 
graphics-based  mode  to  manipulate  objects,  or  interact  with  an 
environment,  as  displayed  in  two  dimensions  on  a monitor 
(computer  or  workstation)  or  on  a head-mounted-display.  For  the 
purposes  of  manufacturing  applications,  the  more  modest  version 
of  VR  is  acceptable  along  with  the  more  ambitious  version. 

ISO  Initiative  in  Information  and  Interfaces  for  Manufacturing 

The  Intelligent  Systems  Division  (ISD)  and  the  Factory 
Automation  Systems  Division  (FASD)  of  the  Manufacturing 
Engineering  Laboratory  (MEL)  at  NIST  are  concerned  with 
standardizing  information  protocols  and  interface  technologies 
and  techniques  for  manufacturing  technology.  ISD  is  exaunining 
the  human  factors  involved  in  interfaces  for  appropriate 
information  interactions.  NIST  is  interested  in  exploring 
virtual  and  distributed  manufacturing  with  initial  testbed 
efforts  in  the  production  of  discrete  parts.  Primary  target 
areas  include:  Engineering  Design,  Product  Planning,  Process 
Planning,  Factory  Simulation,  Monitoring  and  Control. 

ISD  is  working  with  users  and  developers  to  coordinate 
standard  information  protocols  and  interfaces,  which  will 
eventually  be  reflected  in  the  integration  of  virtual  reality  in 
a manufacturing  freunework.  This  human  factors  initiative  will 
have  a major  impact  in:  information  content  for  each  user; 
presentation  bandwidth  and  fidelity;  presentation  modality  - real 
time  or  non-real  time  and  mixes  of  text,  graphics,  pictures, 
videos,  and  other  sensory  modalities. 

VR  Threads 

Virtual  reality  may  be  able  to  integrate  intelligent 
manufacturing  systems:  products,  processes,  and  enterprises.  The 
potential  impact  of  virtual  reality  may  be  dreunatic.  VR,  which 
will  be  defining  itself  from  the  user  pull  and  the  technology 
push  over  the  next  several  years,  has  several  common  threads: 

* Virtual  reality  provides  new  worlds  for  one  or  more  users. 

Related  terminology  is:  artificial  worlds,  virtual 
environments,  and  synthetic  environments. 
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* Some  VR  systems  attempt  to  give  the  user  a sense  of 
telepresence  by  immersive  sensation  (vision,  sound,  tactile 
(pressure),  haptile  (force).  Virtual  reality  may  apply  to 
single  sites  with  human/machine  interfaces  to  co-located 
computers,  or  to  interconnected,  distributed  systems  at  distant 
sites  linked  through  communications  networks. 

* Virtual  reality  establishes  a fraunework  which  will  ultimately 
integrate  many  areas  of  technology:  human  factors,  computer 
hardware  and  software,  and  communications. 

* Virtual  reality  permits  each  user  to  participate  in  a 
collective  effort. 

Virtual  reality  is  supported  by  enabling  technology  programs 
in  the  following  areas: 

* Communications:  the  National  Information  Infrastructure  (Nil) 

* High  performance  computation:  High  Performance  Computing  and 
Communications  (HPCC) 

* Data  and  interface  standards:  PDES-STEP 

* Hardware  including  head  mounted  displays  (HMDs),  body  suits, 
exoskeletons,  data  gloves,  large  screens 

* Human  factors  studies. 

VR  to  date  has  been  focused  primarily  in  four  application 
areas : 

(1)  Simulation  - which  yields  new  views  and  perspectives  which 
are  unattainable  by  current  methods. 

(2)  Teleconferencing  participation  - for  "meeting  at  a distance," 
with  a sense  of  telepresence,  in  a collaborative,  immersive 
environment  for  joint  activities,  including  discussions, 
interviews,  planning,  and  training 

(3)  Training  and  Education,  including: 

* Integration  of  training  for  personnel  at  distributed  sites 
with  distributed  elements 

* Using  simulations  or  synthetic  environments  to  achieve  cost 
savings  by  not  using  "real  elements;  when  "real"  elements  are 
unavailable;  and  to  protect  personnel  in  training  for  hazardous 
operations  and  environments,  e.g.,  nuclear  environments,  toxic 
contamination,  or  live-fire  conditions 

* Students  using  educational  virtual  reality  toolsets  to  build 
their  own  virtual  worlds,  to  immerse  themselves,  and  to 
interact  in  "what  if?"  situations 

* Students  can  learn  by  interacting  with  unavailable  and  out-of- 
size  components  and  environments,  such  as  observing  a real-time 
manufacturing  process  through  a magnifying  glass. 

(4)  Mutual  participation  in  shared  activities  in  a synthetic 
environment.  The  most  striking  current  participatory 
applications  of  virtual  reality  are  for:  Entertainment  (several 
theme  parks  have  simulator  rides  and  other  are  currently  being 
design  as  entire  virtual  reality  theme  parks);  Military  (to 
test  new  technology  or  tactics,  and  to  train  personnel  at 
distributed  sites  in  synthetic  environments). 
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ENQUIRY  FORM: 

APPLICATION  OF  INFORMATION  AND  INTERFACE  TECHNOLOGY 
FOR  VIRTUAL  AND  DISTRIBUTED  MANUFACTURING  OF  DISCRETE  PARTS 

PURPOSE 

The  purpose  of  this  questionnaire  is  to  determine  a vision 
for  human  factors  in  information  and  Interfaces  needed  to 
integrate  virtual  reality  technologies  into  the  virtual  and 
distributed  manufacturing  of  discrete  parts.  You  are  asked  to 
address:  (1)  a definition  of  the  requisite  information  and 
interfaces  for  virtual  and  distributed  manufacturing 
applications,  (2)  human  factors  approaches  in  information  amd 
interface  technologies,  and  (3)  leveraging  of  efforts  in  specific 
topical  areas.  The  results  of  this  questionnaire  will  allow  the 
Intelligent  Systems  Division  (ISD)  and  the  Manufacturing 
Engineering  Laboratory  (MEL)  of  the  National  Institute  of 
Standards  and  Technology  (NIST)  to  determine  needed  standards  and 
protocols  for  information  and  interfaces.  Robotic  Technology 
Inc.  is  working  on  this  questionnaire  under  contract  to  NIST. 

prcx:edure 

You  have  been  provided  with  brief  background  information 
describing  human  factors  and  virtual  reality  for  manufacturing. 
Please  feuniliarize  yourself  with  this  material  before  answering 
the  questions  using  your  expert  judgment  and  opinion. 

Please  tax  only  this  completed  form  (not  the  background 
information),  as  soon  as  possible,  to  (301 )-983-2509 . If  you  have 
any  questions  or  comments,  please  call:  Dr.  Joe  Iseman,  Robotic 
Technology  Inc.,  ( 301 )-983-2465 . Thank  you  for  your 
participation  and  cooperation. 

YOUR  NAME 

Last  First 

POSITION  or  TITLE  


ORGANI ZATION/ AGENCY, 
ADDRESS 


PHONE 

FAX 


E-Mail 
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1.  With  respect  to  VR,  are  you  a potential:  ( Please  check  one) 
(a)  Developer  [ ] (b)  User  [ ] (c)  Researcher  [ ] 

Comments  (if  any): 


2.  How  much  do  you  know  about  virtual  reality?  (Please 
indicate  with  an  "x"  on  the  scale): 

/ / / / / / 

0 1 2 3 4 5 

Don’t  Know  Much  Very  Knowledgeable 

Comments  (if  any): 


3.  How  would  you  best  characterize  the  field  of  "Virtual 
Reality"?  (Please  indicate  with  an  "x"  on  the  scale): 

/ / / / / / 

0 1 2 3 4 5 

Figment  of  Imagination  Realistic 

Comments  (if  any): 


4.  How  much  do  you  know  about  human  factors /ergonomics  for 
defining  information  presentations  with  computer-interface 
technology  and  techniques?  (Please  indicate  with  an  "x"  on  the 
scale) : 

/ / / / / / 

0 1 2 3 4 5 

Don’t  Know  Much  Very  Knowledgeable 

Comments  (if  any):^ 
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5.  How  would  you  best  characterize  the  areas  of  human  factors/ 
ergonomics  for  defining  information  presentations  with  computer- 
interface  technology  and  techniques?  (Please  indicate  with  an 
"x"  on  the  scale ) : 

/ / / / / / 

0 1 2 3 4 5 

Inadequate  Well  Developed 

Comments  (if  any): 


6.  Are  information  presentations  currently  adequately  embedded 
in  the  manufacturing  process?  (Check  one) 

(a)  Yes  [ ] (b)  No  [ ] (c)  Maybe  [ ] 

Comments  ( i f any ) : 


7.  For  the  kinds  of  manufacturing  activities  with  which  you  are 
familiar,  what  kinds  of  information  would  typical  users  require 
to  perform  their  job  remotely:  (Please  comment  on  as  many 
activities  as  you  wish) 

7a.  Shop  floor  supervisors  


7b.  Machine  operators 


7c.  Inspectors 


7d.  Material  handlers 


7e.  Maintenance  and  repair  personnel 


7f.  Supply  room  personnel 
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79.  Tool  room  personnel 


7h.  CAD-CAM  designers 


71.  Design  engineers 


7j.  Process  planning  engineers 


7k.  Manufacturing  consultants 


71 . Plant  managers 


7m.  Schedulers 


7n.  Schedulers 


7o.  Accountants 


7p.  Management  information  systems  personnel 


7q.  Business  consultants 


7r.  External  environment  personnel 


7s.  Other  (please  specify) 
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8.  Are  you  familiar  with  interactive  text-based  virtual  reality 
information  presentations  (i.e.,  on-line  dungeons  and  dragons 
games ) ? ( Check  one ) 

(a)  Yes  [ ] (b)  No  [ ] (c)  Somewhat  [ ] 

Conments  (if  any): 


9.  When  will  text-based  information  presentations  using  VR- 
computer-interface  technology  first  be  used  in  manufacturing? 
(Please  indicate  with  an  "x"  on  the  scale): 

/ / / / 

1995  2005  2015  2025  >2025 

Comments  (if  any): 


10.  Are  you  familiar  with  interactive  graphics-based  virtual 
reality  information  presentations  (head-mounted  displays,  small 
or  large  screens  in  entertainment)?  (Check  one) 

(a)  Yes  [ ] (b)  No  [ ] (c)  Somewhat  [ ] 

Comments  (if  any): 


1 1 .  When  will  graphics-based  information  presentations  on  VR- 
computer- interface  technology  first  be  of  used  in  manufacturing? 
(Please  indicate  with  an  "x"  on  the  scale): 

/ / / / 

1995  2005  2015  2025  >2025 

Comments  (if  any): 


12.  What  do  you  believe  are  the  most  important  features  of 
information  presentations  with  VR  computer- interface  technology 
for  manufacturing? 
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13.  With  respect  to  VR  graphic  presentations,  which  features  are 
most  important  to  you:  (Please  prioritize:  Most  important  - 1) 

(a)  Visual  quality  [ ] (b)  Refresh  speed  [ ] (c)  Seamless  [ ] 

(d)  Tactile  interactions  [ ] (e)  Other  (specify  below)  [ ] 

Comments  (if  any): 


14.  What  do  you  think  of  the  likelihood  that  information 
structures  and  VR  user  interfaces,  customized  to  support  specific 
users  will  be  useful  and  valuable  in  manufacturing: 


/ / 

P=  0 .2 

Low 

Probability 
Comments  (if  any): 


/ / 


.4  .6 

Moderate 

Probability 


/ / 


.8  1.0 

High  Certain 

Probability 


15.  What  would  be  the  value  added  by  customized  information 
presentations  on  VR  computer-interface  technology  for 
manufacturing?  (Please  indicate  with  an  "x"  on  the  scale): 

/ / / / / / 

0 1 2 3 4 5 

None  Moderate  High 

Comments  ( i f any ) : 


16.  Who  will  drive  information  presentations  and  VR  computer- 
interface  technology  for  manufacturing?  (Check  one) 

(a)  Developers  [ ] (b)  Users  [ ] (c)  Both  [ ] 

Comments  (if  any ) : 
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17.  Who  will  be  primary  funding  sources  for  information 
presentations  with  VR  computer-interface  technology  for 
manufacturing?  (Check  one) 

(a)  Developers  [ ] (b)  Users  [ ] (c)  Government  [ ] 

Comments  (if  any): 


18.  Which  technologies  do  you  believe  must  be  exploited  to  make 
information  presentations  with  VR  computer- interface  technology 
for  manufacturing  workable  and  affordable  to  users? 


19.  Which  industries  or  types  of  processes  do  you  feel  can 
benefit  first  from  information  presentations  with  VR  computer- 
interface  technology  for  manufacturing  applications? 


20.  How  would  you  rank  the  necessary  growth  of  the  following 
technological  factors  for  the  integration  of  information 
presentations  on  VR  computer- interface  technology  into 
manufacturing?  (Please  indicate  with  an  *'x"  on  the  scales  for 
each  factor): 

20a.  Computer  hardware?  / / / / / / 

0 1 2 3 4 5 

None  A Great  Deal 

20b.  Communications  systems?  / / / / / / 

0 1 2 3 4 5 

None  A Great  Deal 

20c.  Software?  / / / / / / 

0 1 2 3 4 5 

None  A Great  Deal 

20d.  Human  Factors?  / / / / / / 

0 1 2 3 4 5 

None  A Great  Deal 

Comments  (in  any): 
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21.  How  well  can  standards,  open  architectures,  and 
interoperability  be  achieved  in  information  presentations  with  VR 
computer-interface  technology  to  define  a framework  across 
multiple  vendors  of  hardware  and  software  and  users?  (Please 
indicate  with  an  "x"  on  the  scale): 

/ / / / / / 

0 1 2 3 4 5 

Poor  Excellent 

Comments  (if  auiy): 


22.  Does  special  equipment  now  required  for  realism  - head 
mounted  displays,  data  gloves  - reduce  or  enhance  the  ability  of 
the  user?  (Please  indicate  with  an  ”x"  on  the  scale): 

/ / / / / / 

0 1 2 3 4 5 

Reduce  Enhance 

Comments  (if  any): 


23.  How  well  can  VR  tools  for  the  information  presentation  with 
VR  computer-interface  technology  be  integrated  with  manufacturing 
technology  tools?  (Please  indicate  with  an  "x"  on  the  scale): 

/ / / / / / 

0 1 2 3 4 5 

Poor  Very  Well 

Comments  ( i f any ) : 


24.  How  would  you  rate  the  applicability  of  the  fidelity  of 
information  presentation  achieved  with  VR  computer-interface 
technology  in  manufacturing  technology  for  discrete  part 
production  in  each  of  the  following  areas?  (Please  indicate  with 
an  "x"  on  the  scales ) : 

24a.  Training?  / / / / / / 

0 1 2 3 4 5 

Poor  Excellent 

24b.  Product  design?  / / / / / / 

0 1 2 3 4 5 

Poor  Excellent 
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24c.  Process  design? 


24d.  Process  validation? 


24e.  Enterprise  integration? 


Comments  ( in  any ) : 


/ / / 

0 1 2 

Poor 

/ / / / 

0 12  3 

Poor 

/ / / / 

0 12  3 

Poor 


/ / 

4 5 

Excellent 


/ 


/ 


4 5 

Excellent 


25.  Can  the  US  manufacturing  industry  afford  to  apply  high- 
fidelity,  customized  information  presentations  with  VR  computer- 
interface  technology?  (Please  indicate  with  an  ”x"  on  the 
scale) : 

/ / / / / / 

0 1 2 3 4 5 

Can  not  Can 

Comments  (in  any): 


26.  How  much  will  cost  of  equipment  to  support  a high  quality 
graphical  and  interaction  interfaces  (future  generations  of 
workstations,  head-mounted  displays,  and  data  gloves)  effect 
acceptance  of  a virtual  workplace?  (Please  indicate  with  an 
"x”  on  the  scale): 

/ / / / / / 

0 1 2 3 4 5 

None  A Great  Deal 

Comments  (in  any): 


27.  Do  you  know  of  direct  requirements  for  VR  interface 
presentations  for  any  manufacturing  systems?  (Please  check  one) 

(a)  Yes  [ ] (b)  No  [ ] (c)  Maybe  [ ] 

Comments  (if  any): 
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28.  Will  information  presentation  with  VR  computer-interface 
technology  have  an  overall  worth  in  integrating  companies, 
enterprises,  or  industries  more  intelligently  into  the 
manufacturing  process?  (check  one) 

(a)  Yes  [ ] (b)  No  [ ] (c)  Maybe  [ ] 

Comments  (if  any): 


29.  Will  information  presentation  with  VR  computer- interface 
technology  integrate  vendors /suppliers  to  a greater  extent  into 
the  manufacturing  process?  (Please  check  one) 

(a)  Yes  [ ] (b)  No  [ ] (c)  Maybe  [ ] 

Comments  (if  any): 


30.  Please  make  any  additional  comments  below: 
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SURVEY  REPORT 

Activities  Preliminary  To  Convening  A Workshop 
On  The  Application  Of  Virtual  Reality  To  Manufacturing 
Results  Of  A Survey  (August  94) 


m t«fi(3-ilV  \<5  aoJtSaoilqqA  oAT  no 

S«M^wiS.V  >ie>».-X5^  Ari’SO  aiituatBSV 


*■ 


a 


3 

^ J 


CONTENTS 


Section  Page 

1.0  PURPOSE  1 

2.0  THE  SURVEY  1 

2 . 1 Motivation  1 

2.2  Survey  Results  4 

Figures  34 

APPENDIX  A:  Enquiry  Form  66 

APPENDIX  B:  Survey  Respondents  81 


^ r> 


FINAL  REPORT 


1 


1 . 0 PURPOSE 

The  purpose  of  this  project  was  to  perform  certain 
activities  preliminary  to  convening  a workshop  on  the  application 
of  virtual  reality  (VR)  to  manufacturing.  In  the  course  of  the 
work,  we  performed  a literature  search  to  determine  the  state-of- 
the-art  of  virtual  reality  as  it  relates  to  information  and 
interface  technology  for  virtual  and  distributed  manufacturing  of 
discrete  parts.  We  found  nothing  relevant  to  the  subject  of 
interest.  The  main  task  was  to  perform  a survey  to  determine 
some  of  the  issues  in  the  application  of  information  and 
interface  technology  for  virtual  and  distributed  manufacturing  of 
discrete  parts. 

2 . 0 THE  SURVEY 

An  enquiry  form  was  designed  as  given  in  Appendix  A.  About 
half  the  recipients  of  the  survey  responded,  a satisfactory 
response  rate.  The  respondents  are  listed  in  Appendix  B. 

2 . 1 Motivation 

The  National  Institute  of  Standards  and  Technology  (NIST)  is 
supporting  new  initiatives  in  manufacturing  technology  which 
promise  to  have  a major  impact  on  the  national  economy.  The 
Intelligent  Systems  Division  (ISD)  of  NIST  is  exploring  the 
potential  applications  of  human  information  and  interface 
technology  for  virtual  and  distributed  manufacturing.  The 
purpose  of  the  enquiry  is  to  determine  some  of  the  issues  in 
human  factors  and  virtual  reality  for  manufacturing  from  the 
user's  perspective  and  to  build  a new  vision  for  manufacturing 
systems. 

Key  questions  relate  to  the  attributes  of  human  information 
interactions  through  lO  interfaces  and  their  impact  on  potential 
leveraging  of  manufacturing  technology.  Who  needs  to  look  at 
information?  What  kinds  of  information  are  appropriate  for  each 
user  to  support  real-time,  collaborative  decision-making?  What 
differences  are  there  in  information  needed  for  different  users  - 
individually  or  in  new  collaborative  process  within  a "virtual" 
presentation  environment?  What  are  the  important  issues  in  how 
users  choose  to  view  information  - both  text  or  graphics?  What 
human  factors  issues  and  design  considerations  should  be  included 
in  a VR  computer-interface?  How  can  the  information  content  and 
presentation  best  be  approached  with  VR  computer-interface 
technologies? 

Manufacturing  technology  is  currently  a focus  of  national 
initiatives  sponsored  by  numerous  government  organizations  and 
industrial  consortia.  There  is  a commitment  of  funding  by 
programs  in  the  Department  of  Commerce  (Advanced  Technology 
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Program  [ATP]),  Department  of  Defense  (Technology  Reinvestment 
Program  [TRP]),  as  well  as  the  Department  of  Energy,  NASA,  and 
other  Federal  and  state  agencies.  All  of  these  programs  address 
the  necessity  of  boosting  US  manufacturing  capabilities. 

In  this  regard  NIST  has  been  given  an  expanded  role  in 
standardizing  technological  infrastructures  and  applications  in 
such  programs  as: 

* Manufacturing  Research  Technology  Centers  (METCs) , which  have 
become  major  regional  technology  resources 

* The  Automated  Manufacturing  Research  Facility  (AMRF) , an  open 
architecture  test-bed  for  the  automated  manufacture  of  discrete 
parts . 

* The  PDES-STEP  (Product  Data  Exchange  using  STEP  - Standard  for 
the  Exchange  of  Product  Model  Data)  Program  for  standardizing 
manufacturing  data  representations. 

Virtual  reality  is  rapidly  becoming  technically  and 
economically  feasible.  VR  has  many  definitions  and  formats.  At 
the  ambitious  end  of  the  spectrum,  VR  allows  a user  to  have  the 
illusion  that  he  is  immersed  in  a synthetic  or  remote 
environment.  He  experiences  sensory  information,  such  as  three- 
dimensional  vision  and  binaural  sound,  that  is  generated  by  a 
computer  or  originates  from  remote  sensors  (or  is  a combination 
of  the  two  sources) . He  interacts  kinesthetically  with  computer 
generated  or  teleoperated  objects.  At  the  less  ambitious  end  of 
the  spectrum,  VR  allows  a user  to  operate  in  a text-based 
interactive  modes  (i.e.,  on-line  dungeons  and  dragons)  or  a 
graphics-based  mode  to  manipulate  objects,  or  interact  with  an 
environment,  as  displayed  in  two  dimensions  on  a monitor 
(computer  or  workstation)  or  on  a head-mounted-display.  For  the 
purposes  of  manufacturing  applications,  the  more  modest  version 
of  VR  is  acceptable  along  with  the  more  ambitious  version. 

The  Intelligent  Systems  Division  (ISD)  and  the  Factory 
Automation  Systems  Division  (FASD)  of  the  Manufacturing 
Engineering  Laboratory  (MEL)  at  NIST  are  concerned  with 
standardizing  information  protocols  and  interface  technologies 
and  techniques  for  manufacturing  technology.  ISD  is  examining 
the  human  factors  involved  in  interfaces  for  appropriate 
information  interactions.  NIST  is  interested  in  exploring 
virtual  and  distributed  manufacturing  with  initial  testbed 
efforts  in  the  production  of  discrete  parts.  Primary  target 
areas  include:  Engineering  Design,  Product  Planning,  Process 
Planning,  Factory  Simulation,  Monitoring  and  Control. 

ISD  is  working  with  users  and  developers  to  coordinate 
standard  information  protocols  and  interfaces,  which  will 
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eventually  be  reflected  in  the  integration  of  virtual  reality  in 
a manufacturing  framework.  This  human  factors  initiative  will 
have  a major  impact  in:  information  content  for  each  user; 
presentation  bandwidth  and  fidelity;  presentation  modality  - real 
time  or  non-real  time  and  mixes  of  text,  graphics,  pictures, 
videos,  and  other  sensory  modalities. 

Virtual  reality  may  be  able  to  integrate  intelligent 
manufacturing  systems:  products,  processes,  and  enterprises.  The 
potential  impact  of  virtual  reality  may  be  dramatic.  VR,  which 
will  be  defining  itself  from  the  user  pull  and  the  technology 
push  over  the  next  several  years,  has  several  common  threads: 

* Virtual  reality  provides  new  worlds  for  one  or  more  users. 
Related  terminology  is:  artificial  worlds,  virtual 
environments,  and  synthetic  environments. 

* Some  VR  systems  attempt  to  give  the  user  a sense  of 
telepresence  by  immersive  sensation  (vision,  sound,  tactile 
(pressure) , haptile  (force) . Virtual  reality  may  apply  to 
single  sites  with  human/machine  interfaces  to  co-located 
computers,  or  to  interconnected,  distributed  systems  at  distant 
sites  linked  through  communications  networks. 

* Virtual  reality  establishes  a framework  which  will  ultimately 
integrate  many  areas  of  technology:  human  factors,  computer 
hardware  and  software,  and  communications. 

* Virtual  reality  permits  each  user  to  participate  in  a 
collective  effort. 

Virtual  reality  is  supported  by  enabling  technology  programs 
in  the  following  areas: 

* Communications:  the  National  Information  Infrastructure  (Nil) 

* High  performance  computation:  High  Performance  Computing  and 
Communications  (HPCC) 

* Data  and  interface  standards:  PDES-STEP 

* Hardware  including  head  mounted  displays  (HMDs) , body  suits, 
exoskeletons,  data  gloves,  large  screens 

* Human  factors  studies. 

VR  to  date  has  been  focused  primarily  in  four  application 
areas: 

(1)  Simulation  - which  yields  new  views  and  perspectives  which 
are  unattainable  by  current  methods. 
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(2)  Teleconferencing  participation  - for  ’‘meeting  at  a distance,*' 
with  a sense  of  telepresence,  in  a collaborative,  immersive 
environment  for  joint  activities,  including  discussions, 
interviews,  planning,  and  training 

(3)  Training  and  Education,  including: 

* Integration  of  training  for  personnel  at  distributed  sites 
with  distributed  elements 

* Using  simulations  or  synthetic  environments  to  achieve  cost 
savings  by  not  using  ’’real  elements;  when  ’’real”  elements  are 
unavailable;  and  to  protect  personnel  in  training  for  hazardous 
operations  and  environments,  e.g.,  nuclear  environments,  toxic 
contamination,  or  live-fire  conditions 

* Students  using  educational  virtual  reality  toolsets  to  build 
their  own  virtual  worlds,  to  immerse  themselves,  and  to 
interact  in  ”what  if?”  situations 

* Students  can  learn  by  interacting  with  unavailable  and  out-of- 
size  components  and  environments,  such  as  observing  a real-time 
manufacturing  process  through  a magnifying  glass. 

(4)  Mutual  participation  in  shared  activities  in  a synthetic 
environment.  The  most  striking  current  participatory 
applications  of  virtual  reality  are  for:  Entertainment  (several 
theme  parks  have  simulator  rides  and  other  are  currently  being 
design  as  entire  virtual  reality  theme  parks) ; Military  (to 
test  new  technology  or  tactics,  and  to  train  personnel  at 
distributed  sites  in  synthetic  environments) . 

2.2  Survey  Results 

The  results  of  the  survey  are  presented  in  the  context  of 
the  enquiry  form.  All  of  the  figures  (graphs)  are  clustered  at 
the  end  of  this  section. 

1.  With  respect  to  VR,  are  you  a potential:  ( Please  check  one) 

(a)  Developer  [ 8 ] (b)  User  [ 2 ] (c)  Researcher  [ 24  ] 

n=34  (some  gave  more  than  one  response) 

The  majority  of  respondents  were  researchers,  as  shown  in 
Figure  1.  Comments  from  the  respondents  included: 

(1)  Anticipate  the  use  of  VR  technology  for  planning  and 
control  applications  in  manufacturing. 

(2)  Mostly  vision  and  software  research,  some  graphics. 

(3)  Have  $1  M grant  from  NSF  to  study  educational 
implications  of  VR. 
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(4)  UTRC  would  be  a combination  of  all  three. 

(5)  We  are  researching  text  and  graphics-based  VR  for 
distributed  manufacturing  systems. 

(6)  Particular  focus  in  manufacturing  applications  (to 
include  design  of  prod/process) . 

(7)  I develop  S/W  for  VR  and  perform  extensive  research  in 
its  use  in  education,  training,  and  data  visualization. 

(8)  All  three  choices  apply  - I've  worked  on  training  PD 
with  virtual  images,  currently  doing  basic  VR  research. 

(9)  We've  built  our  own  system. 

(10)  Currently  manage  a seven-person  VR  research  team. 

(11)  Our  effort  involves  both  research  and  development. 

2.  How  much  do  you  know  about  virtual  reality?  (Please 
indicate  with  an  "x"  on  the  scale) : 

0 2 6 11  10 

/ / / / / / 

0 1 2 3 4 5 

Don't  Know  Much  Very  Knowledgeable 

n Mean Sta.  Dev. Range 

29  3.50  1.03  0-5 

Most  of  the  respondents  are  quite  knowledgeable  about  VR,  as 
illustrated  in  Figure  2.  Comments  include: 

(1)  UTRC  has  spent  the  last  three  years  developing  a VR  lab. 

(2)  MSE  from  the  advanced  interfaced  program  at  UW, 
currently  in  PhD  program. 

(3)  I was  involved  in  original  studies  at  NASA-ARC  in  1986. 

3.  How  would  you  best  characterize  the  field  of  "Virtual 
Reality"?  (Please  indicate  with  an  "x"  on  the  scale) : 

1 1 4 10  13 

/ / / / / / 

0 1 2 3 4 5 

Figment  of  Imagination  Realistic 

n Mean Sta.  Dev. Range 

29  3.60  1.02  0-5 

The  great  majority  of  respondents  believe  that  VR  is  a 
feasible  endeavor,  a "real"  field,  as  shown  in  Figure  3.  But 
there  are  some  cynics.  Comments  include: 

(1)  Some  real  meat,  mostly  hype. 

(2)  Except  for  flight  simulators,  VR  has  been  a lot  of  hype. 
For  VR  to  succeed  it  needs  practical  applications 

(3)  VR  is  definitely  within  reach,  but  technical,  political. 
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and  economic  forces  make  for  quite  a dynamic  environment. 

(4)  That's  a funny  question.  By  definition,  VR  is  a 
"figment  of  imagination"  that  is  very  "realistic."  I'll  assume, 
however,  that  you're  asking  about  the  promise  of  the  field 1 

(5)  Concern  is  that  it  is  used  in  appropriate  situations. 

(6)  The  range  of  "system"  spans  this  scale  0->3.5  in  my 
judgment. 

(7)  Projected  rapid  improvement. 

(8)  Realistic,  but  still  limited. 

(9)  Competitively  intensive,  RT  critical  and  therefore 
computer  HW  is  expensive. 

(10)  I'm  not  sure  what  you're  getting  at — the  technology 
already  exists.  The  press  is  quite  misguided. 

(11)  But  very  overhyped,  leading  to  unrealistic  expectations 
& disappointment. 

4.  How  much  do  you  know  about  human  factors/ergonomics  for 
defining  information  presentations  with  computer-interface 
technology  and  techniques?  (Please  indicate  with  an  "x"  on  the 
scale) : 

0 37  12  7 

/ / / / j / 

0 1 2 3 4 5 

Don't  Know  Much  Very  Knowledgeable 

n Mean Sta.  Dev. Range 

29  3.14  1.06  0-5 

Host  of  the  respondents  are  reasonably  or  very  knowledgeable 
about  the  human  factors  relevant  to  the  topic  at  hand,  as 
depicted  in  Figure  4.  Comments  include: 

(1)  We  have  a consortium  of  75+  small  and  large  companies. 

In  addition,  we  have  coordinated  with  a couple  hundred  machine 
shops  to  research  and  document  issues  related  to  information 
views  and  human  factors/ergonomics. 

(2)  Few  IE  courses  and  psychology  seminars. 

(3)  I have  a "seat-of -pants"  reaction,  but  no  formal  insight 
into  human  factors/ergonomics. 

(4)  We  will  be  moving  into  human  factors  in  the  next  FY. 

(5)  Very  strong  knowledge  of  auditory  display;  less  so 
regarding  traditional  visual  displays,  though  fairly  good  with  VR 
visual  displays. 
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5.  How  would  you  best  characterize  the  areas  of  human  factors/ 
ergonomics  for  defining  information  presentations  with  computer- 
interface  technology  and  techniques?  (Please  indicate  with  an 
•*x”  on  the  scale)  : 

2 11  7 6 1 

/ / / / ! / 

0 1 2 3 4 5 

Inadequate  Well  Developed 

n Mean Sta,  Dev. Range 

27  2.13  0.90  0-5 

As  Figure  5 shows,  most  of  the  respondents  feel  that  the 
technology  is  lacking  and  needs  much  more  work,  although  there 
are  a few  optimists.  Comments  include: 

(1)  Great  strides  made,  still  a long  way  to  go. 

(2)  Few  well  known  sources  of  information  that  provide 
guidelines  to  developers. 

(3)  Well-developed  for  2-D  displays  - we  have  much  to  do  in 
3-D/VR  displays  - especially  multi-sensory  displays 

(4)  The  bottleneck  in  VR  isn't  hardware  or  software,  it's 
the  human  factors  of  interfaces. 

(5)  Traditional  factors  may  not  apply  to  VR. 

(6)  It  is  strong  in  evaluative  terms,  less  so  in  generative 
terms.  I've  worked  on  analytic  modeling  (pm  forms). 

(7)  It  needs  more  **Kan-sei”  design  engineering  principles  & 
approaches  to  it. 

6.  Are  information  presentations  currently  adequately  embedded 
in  the  manufacturing  process?  (Check  one) 

(a)  Yes  [ 1 ] (b)  No  [ 17  ] (c)  Maybe  [ 9 ] 

n=27 

Only  one  lone  respondent  is  certain  that  there  are  adequate 
presentations  of  information  embedded  in  the  manufacturing 
process,  as  graphically  illustrated  in  Figure  6.  Comments 
include: 

(1)  Interfaces  to  most  control  systems  are  text  or  at  best, 
iconic. 

(2)  What  I've  seen  is  changing. 

(3)  Little  use  of  visualization  and  multisensory 
presentation  of  data,  telepresence. 

(4)  Rarely! 

(5)  Do  we  even  know  what  information  to  present,  and  when  to 
present  it? 

(6)  Some  companies  are  already  quite  sophisticated,  judging 
from  what  I hear  about  Boeing,  for  example. 
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(7)  From  what  I know,  only  a few  hi-tech  industries  have 
begun  to  use  information  technology. 

(8)  Textual  (PC)  in  most  Manufacturing  Processes.  Typically 
only  Design/Engineering  have  any  workstation  class  computers. 

(9)  There  are  various  examples  of  such  techniques. 

(10)  It  depends  on  applications. 

7.  For  the  kinds  of  manufacturing  activities  with  which  you  are 
familiar,  what  kinds  of  information  would  typical  users  require 
to  perform  their  job  remotely:  (Please  comment  on  as  many 
activities  as  you  wish) . 

Comments  are  given  for  each  subsection,  where  respondents 
recommend  information  spanning  the  VR  spectrum  from  "ambitious" 
to  "non-ambitious" : 


(1)  Hard  to  answer  these  questions  generally — depends  on 
what  is  being  manufactured. 

(2)  Not  familiar  with  manufacturing  activities. 

(3)  I don't  work  in  manufacturing  so  my  knowledge  is 
limited. 

(4)  All  should  have  access  to  all  info  and  stages  with 
priority  and  "paths"  determined  by  the  job. 

(5)  I have  very  little  knowledge  of  the  manufacturing 
process  and  do  not  know  current  state  of  info  displays  in  it. 

7a.  Shop  floor  supervisors  


(1)  Availability  of  resources  (tools,  machines,  people)  as 
well  as  manufacturing  capability. 

(2)  Status  trends  throughput  loads,  performance. 

(3)  Excellent  training,  layout  design  for  ease  of 
supervision,  visual  information. 

(4)  Production  schedules,  "hot"  deliverables,  resource 
status. 

(5)  Schedules,  quality  metrics  for  individuals. 

(6)  Real-time  worker  status,  including  productivity  & 
schedule;  production  status  (versus  orders  and  inventory) . 

(7)  Schemas  and  Scripts.  Process  control  information; 
temporal  (synchronization)  and  social  (coordination)  information 
about  shop  workers. 

(8)  Production  status,  machine  status,  master  production 
schedules. 

(9)  Production,  quality  parts  availability. 

(10)  All  senses. 

(11)  RT  machine-operator-order  status. 

(12)  Scheduling  charts. 

(13)  Job  scheduling. 

(14)  Status  of  work  progressing  at  different  workstations. 
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7b.  Machine  operators 


(1)  Part  requirements  including  adequate  description  of 
feature  and  part  dimensions,  how  to  perform  the  operation,  NC 
program. 

(2)  Machine  and  process  parameters  (high  level  information) , 
routing  sheet  & process  plan. 

(3)  Multiple  views  of  machine/parts  & teleoperation  limited 
to  controls  & feeders. 

(4)  Full  immersive  VR,  including  audio,  tactile,  vision  and 
olfactory  senses;  Real-time  machine  status. 

(5)  Dispatch  and/or  production  schedules,  machine  status, 
quality/metrology . 

(6)  Quality,  production. 

(7)  All  senses. 

(8)  Queue  of  jobs,  kitting-f ixture-stock  location  and 
status,  RT  solid  modeling. 

(9)  Telepresence  techniques. 

(10)  Close  visual  and  kinesthetic  feedback. 

(11)  Process  display  of  ongoing  work  at  their  station. 

7c.  Inspectors  


(1)  Salient  and  important  characteristics  of  part  together 
with  "what  if"  scenarios  depending  upon  inspection  results. 

(2)  Status,  trends,  statistical  analysis. 

(3)  Sensory  data  and  limited  control. 

(4)  Metacognitive  skills  in  problem  solving  (i.e.,  they 
require  procedural  knowledge  about  diagnosis  or  trouble- 
shooting) . 

(5)  Quality  criteria  per  product. 

(6)  Quality,  standards,  specifications. 

(7)  all  senses. 

(8)  High  precision  visual  information. 

(9)  Graphical  displays  of  applied  tests,  views  of  objects. 

7d.  Material  handlers  


(1)  Usage,  cost  failures,  supplies. 

(2)  More  flexible  automation,  better  layout  & material  flow 
path  design,  visual  information. 

(3)  Production  schedules,  routing  sheet. 

(4)  Total  teleoperation. 

(5)  Movement  request  queue,  priority  and  time  available  for 
completion. 
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(6)  Production,  parts  availability,  parts  location. 

(7)  Telepresence,  video/robot  end  effector. 

(8)  Diagrams,  blow-ups,  parts  drawings. 

(9)  Close  visual  and  kinesthetic  feedback. 

7e.  Maintenance  and  repair  personnel  


(1)  Machine  performance  requirements  and  adequate  repair 
manuals. 

(2)  Ease  of  accessibility  of  equipment,  better  layout 
design,  text,  graphics  information. 

(3)  Total  teleoperation. 

(4)  Not  possible  except  for  remote  diagnosis. 

(5)  Procedural  information  with  mental  models,  good 
graphics;  sometimes  semantic  information  (systems  knowledge) . 

(6)  Machine  parts  availability,  machine  specifications. 

(7)  Visual,  aural,  tactile,  force. 

(8)  Pm  schedule 

(9)  Close  visual  and  kinesthetic  feedback. 

(10)  Results  of  diagnostic  tests,  access  to  virtual  manuals. 

7f.  Supply  room  personnel  


(1)  Materials/ tooling  requirements  and  alternatives  which 
can  be  substituted  as  required. 

(2)  Trends,  usage  shelf  life. 

(3)  Better  material  tracking,  streamlining  of  material  flow, 
visual  & auditory  Information. 

(4)  Total  teleoperation. 

(5)  Simple  displays  of  quantity  versus  orders;  Real-time 
displays  of  robot  status. 

(6)  Job  and  associated  tools/materials/fixture  requirements. 

(7)  Parts  availability,  location. 

(8)  Vision. 

(9)  Job  orders,  production  schedule. 

(10)  Inventory,  demand  data,  supplier  data,  specs/parts. 

(11)  Visual  information. 

(12)  Inventory,  images  of  items,  item  history  & location. 

7g.  Tool  room  personnel  


(1)  Expected  performance  of  specified  tooling. 

(2)  Specifications. 

(4)  Job  orders,  kitting  list. 
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7h.  CAD-CAM  designers 


(1)  Communicate  with  design  engineers  and  process  planning 
engineers  to  optimize  design. 

(2)  Remote  from  what?  All  they  need  is  workstation  and 
"contact"  with  engineers. 

(3)  3D  representations  of  objects  tied  to  physics-based 
simulations  and  task-oriented  control  systems. 

(4)  Need  materials,  data/user  requirements /gov't 
constraints/cost  trade-off  data/ environmental  (use)  data/... 

(5)  Specifications. 

(6)  Design,  tool/fixture,  MT  capabilities. 

(7)  Flexible  interactive  graphics. 

(8)  Material  views,  process  visualizations,  animation, 
selected  transformations  of  objects. 

7i.  Design  engineers  


(1)  Communicate  with  CAD/CAM  and  process  planners. 

(2)  Tools  for  design  optimization  interfaced  with  VR, 
graphics,  text  information. 

(3)  Remote  from  what?  all  they  need  is  workstation  and 
"contact"  with  engineers. 

(4)  Functional  requirements.  Users  needs.  Human  factors 
information  relevant  to  interface  design.  Anthropometric  data  on 
humans . 

(5)  Feasibility,  production  process. 

(6)  Flexible  interactive  graphics. 

(7)  All  of  the  above,  plus  immediate  access  to  "virtual 
tests"  & selected  calculations  or  designs. 

7j.  Process  planning  engineers  


(1)  Detailed  specs  and  design  intent.  Communicate  with 
design  and  CAD/ CAM  engineers. 

(2)  Past  data  (stock,  intermediate  part  representation,  and 
final  part  representation) , resources  processing  capability  and 
availability. 

(3)  Feasibility,  production  process. 

(4)  MT  capabilities  & availability  process-feature  mapping. 

7k.  Manufacturing  consultants  
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(1)  User  requirements, 

(2)  Varies,  depending  upon  specific  expertise. 

71.  Plant  managers  


(1)  Resource  availability,  manufacturing  and  delivery 
schedules. 

(2)  Production  schedules  (projected  and  real-time) . 

(3)  High  level  process  flow  displays,  tied  to  sales 
projections,  delivery  schedules,  and  financial  status. 

(4)  material,  processes,  manpower,  money. 

7m.  Schedulers 


(1)  Machine/resource  requirements,  delivery  requirements, 
and  alternate  capabilities. 

(2)  Process  plan,  resource  availability. 

(3)  Just  hard  data  and  access. 

(4)  Employee  status  information;  temporal  information  on  all 
systems;  a synchronization  matrix. 

(5)  Time  and  material. 

(6)  MT  availability  & capabilities,  job  orders,  process 
plans. 

7n.  Schedulers 


(1)  Just  hard  data  and  access. 

(2)  Time  and  material. 

(3)  MT  availability  & capabilities,  job  orders,  process 
plans. 

7 o . Accountants 


(1)  Resource  consumption  and  actual  cost  drivers  to  produce 
product. 

(2)  Just  hard  data  and  access. 

(3)  Access  to  purchase  order  files,  shipping  and  receiving 
info,  on-line  tax  code,  access  to  pending  corporate  decisions. 

(4)  Activity  based  costing/management. 

7p.  Management  information  systems  personnel 
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(1)  Human  factors-type  data,  modeling. 

(2)  Ability  to  access  all  information. 

7q.  Business  consultants  


(1)  Varies  depending  upon  specific  expertise. 
7r.  External  environment  personnel  


(1)  Field  information,  e.g.,  outside  plant  (telephone) 
engineers  need  information  on  existing  cable  and  potential 
development  (housing  tract,  major  hotels) . 

(2)  Job  status  (customers) , inventory  status  & pending 
orders  (suppliers) . 

7s.  Other  (please  specify)  


(1)  I believe  each  of  these  occupations  present  situations 
where  VR  would  prove  a good  alternative.  My  own  area  CAD  is  a 
good  candidate.  Activities  where  teleconf erenced  distributed  (or 
network)  VR  comes  into  its  own  would  be  the  first  to  accept  VR 
interfaces . 


8.  Are  you  familiar  with  interactive  text-based  virtual  reality 
information  presentations  (i.e.,  on-line  dungeons  and  dragons 
games)?  (Check  one) 

(a)  Yes  [ 18  ] (b)  No  [ 3 ] (c)  Somewhat  [ 8 ] 

n=29 

As  Figure  7 shows,  a large  majority  is  at  least  somewhat 
familiar  with  interactive  text-based  VR  information 
presentations,  although  some  do  not  consider  this  type  of 
presentation  to  be  in  the  realm  of  VR.  Comments  include: 

(1)  I would  not  call  these  VRI 

(2)  Have  written  articles  on  MUDs,  MUOs,  etc.  [multi- 
user. . . ] . 

(3)  We  are  currently  working  with  text-based  VR  software  to 
build  prototype  for  machine  shop  management. 

(4)  Little  “serious”  use  at  this  time.  Only  5-10 
educational  systems  in  place. 

(5)  Nothing  against  MUDs,  but  they  are  not  virtual  reality, 
and  calling  them  text-based  VR  probably  stretches  ”VR”  beyond 


FINAL  REPORT 


14 


usefulness. 

(6)  Generally,  I consider  these  to  be  quite  poor. 

9.  When  will  text-based  information  presentations  using  VR- 
computer-interface  technology  first  be  used  in  manufacturing? 
(Please  indicate  with  an  ”x**  on  the  scale)  : 

17  4 0 0 

/ / / / 

1995  2005  2015  2025  >2025 

n Mean Sta.  Dev. Range 

21  2001  4.68  1995-2025+ 


The  respondents  foresee  text-base  VR  for  manufacturing 
available  by  the  year  2001,  as  shown  in  Figure  8,  although  some 
do  not  consider  this  to  be  VR  or  a suitable  technique.  Comments 
include: 

(1)  I am  planning  to  work  collaboratively  with  Motorola  Inc 
on  such  a project. 

(2)  I do  not  advocate  this  first  introduction  of  VR  to 
manufacturing  to  be  based  on  textual  information. 

(3)  I'm  not  convinced  this  kind  of  presentation  is 
appropriate  as  a stand-alone  solution. 

(4)  Assume  you  mean  commercial  application  vs.  research. 

(5)  Educated  guess  based  on  the  rate  of  progress  of 
computing  applications. 

(6)  I am  not  sure  they  should  ever  be  used  when  more 
powerful  displays  are  economic  and  available. 

(8)  Instruction  manuals. 

(9)  The  phrase  is  too  general  to  be  very  meaningful. 

(10)  Never:  Graphics-based  systems  are  already  beginning  to 
overtake  them. 

(11)  The  question  does  not  make  sense  to  me. 

(12)  Nils  Bohr  said  "Prediction  is  difficult,  especially  the 
future. " 

(13)  They  are  already  being  tested  in  Japan. 

(14)  Soonest  applications  will  be  in  planning,  rather  than 
actual  operations. 

10.  Are  you  familiar  with  interactive  graphics-based  virtual 
reality  information  presentations  (head-mounted  displays,  small 
or  large  screens  in  entertainment)?  (Check  one) 

(a)  Yes  [ 25  ] (b)  No  [ 0 ] (c)  Somewhat  [ 4 ] 

n=29 


The  respondents  are  all  at  least  somewhat  familiar  with 
interactive  graphics-based  VR,  as  indicated  in  Figure  9. 
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Comments  Include: 

(1)  Current  technology  has  limitation  based  on  size  and 
time/duration  user  can  sustain  attention  and  necessary  level  of 
performance. 

(2)  wide  range  of  capabilities,  limitations,  and  cost. 

11.  When  will  graphics-based  information  presentations  on  VR- 
computer-interf ace  technology  first  be  of  used  in  manufacturing? 
(Please  indicate  with  an  ”x*'  on  the  scale)  ; 

21  3 1 0 

/ / / / 

1995  2005  2015  2025  >2025 

n Mean Sta.  Dev. Range 

25  2001  5.61  1995-2025+ 

As  with  text-based  technology^  2001  is  the  expected  year  of 
availability  of  graphics-based  VR  for  manufacturing,  as  shown  in 
Figure  10,  although  some  say  it  is  available  now.  Comments 
include: 

(1)  My  forthcoming  project  with  Motorola  Inc.  will  hopefully 
result  in  a prototype  implementation. 

(2)  Unsure,  too  many  other  underlying  technologies  are  in 
the  critical  path  of  any  introduction  of  VR  to  manufacturing. 

(3)  Limited  information  in  a few  years  (not  difficult) . 

(4)  Evolution  of  "through  the  monitor  window"  CAD  into  all 
aspects  of  manufacturing. 

(5)  Cost  is  a major  player  here  ...  plus  processing  power 
and  bandwidth. 

(6)  This  is  an  area  of  research/application  we  are  involved 

with. 

(7)  As  above  [educated  guess]  though  sooner. 

(8)  I am  aware  of  a small  number  of  existing  applications. 

(9)  I read  that  Boeing  has  worked  out  assembly  steps  with 
such  technology. 

(10)  Boeing  is  using  such  for  the  777. 

(11)  They  already  are — Boeing. 

(12)  May  not  make  sense  to  use  this  technology  in 
manufacturing 

(13)  I can  comment  on  state  and  progress  of  VR  technology, 
but  not  the  needs  and  plans  of  manufacturers.  If  you  include 
automobile  companies,  for  example,  the  answer  is  1994  (now) . 

(14)  Should  see  this  introduced  in  certain  areas  before  the 
end  of  the  decade. 

(15)  For  systems  (manufacturing)  of  even  moderate 
complexity. 

12.  What  do  you  believe  are  the  most  important  features  of 
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information  presentations  with  VR  computer- interface  technology 
for  manufacturing? 


Responses  varied  greatly,  but  a core  characterisitc  seems  to 
be  the  presentation  of  information  such  that  the  user  can  absorb 
a maximum  of  information  in  a minimum  time.  Comments  include: 

(1)  More  accurately  decision  making  in  a multi  objective 
trade-off  situation. 

(2)  Ability  to  customize  interface  to  users. 

(3)  That  the  most  appropriate  information  be  presented  with 
the  least  amount  of  tedium  in  the  operator  request  operation,  and 
in  the  easiest  train-of-thought  viewing  format. 

(4)  Good  displays/ interfaces  and  proper  data  being 
displayed. 

(5)  Visualization,  multisensory  presentation  of  data, 
telepresence. 

(6)  Collaborate,  access  to  information  data  base  in  seamless 
fashion,  and  integration  of  manufacturing  with  other  departments 
(engineering,  design,  service,  etc.) 

(7)  The  variety  of  information  or  data  views  and 
representations  for  manipulation. 

(8)  Their  ability  to  present  large  amounts  of  information  in 
ways  that  are  easily  assimilated. 

(9)  That  the  information  be  of  such  content  and  in  such  a 
structure  or  representation  that  it  significantly  aid  the 
problem-solver  (user)  in  the  manufacturing  decision  he  or  she 
must  make  at  this  point  in  time. 

(10)  Hands-free,  ability  to  use  effectively  for  several 
hours;  rugged;  lower  cost. 

(11)  Granularity  of  information — the  ability  to  control  this 
for  VR  applications. 

(12)  Allowing  cooperative/shared  views  with  other  people. 

(13)  Robust  ”scaled**  to  task,  economic,  provide  true  "value- 
added”  compared  with  previous  technology  uses. 

(14)  Planning  and  process  simulation;  information  exchange. 

(15)  Spatial  representation,  registered  to  the  user's 
eyepoint,  and  then  augmented  with  functional  information. 

(16)  Cooperative  sharing  of  the  workspace;  separate 
components  integrated  across  distributed  sites. 

(17)  Clear,  intelligible  displays;  unobtrusive  hardware. 

(18)  RT  data,  inventory  tracking,  production  priority  & 
control . 

(19)  Non-encumbering  displays/ input  devices.  Lots  of  on- 
line, interactive  help  and  navigation  guidance. 

(20)  Real-time  interactivity,  multisensory  input/output 
choice  of  representation. 
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(21)  Resolution;  display  size  (field  of  view) . 

(22)  Depends  somewhat  on  the  manufacturing  task.  But 
primarily  the  flexible  and  rich  visual  feedback.  (But  not 
necessarily  immersive  characteristics) . 

(23)  Feedback  from  applied  controls;  Seamless  switching 
between  monitoring  & control  to  accessing  needed  information. 

13.  With  respect  to  VR  graphic  presentations,  which  features  are 
most  important  to  you:  (Please  prioritize:  Most  important  = 1; 
least  important  = 5) 

(a)  Visual  quality  [ ] (b)  Refresh  speed  [ ] (c)  Seamless  [ ] 

(d)  Tactile  interactions  [ ] (e)  Other  (specify  below)  [ ] 


a b 


1 11  8 

2 4 7 

3 7 4 

4 12 

5 0 0 

n = 27  24 


c d e 


3 2 3 

11  0 2 

2 4 2 

6 7 2 

0 4 2 

19  15  6 


Figure  11  displays  the  respondents 's  prioritization,  1 
through  5,  for  each  of  the  VR  features  listed.  The  results  are 
also  shown  above  in  a matrix  format.  Quality,  refresh  speed,  and 
seamlessness  were  deemed  the  most  important  of  the 
characteristics,  although  others  wished  to  include  resolution, 
sound,  and  other  features  to  be  considered  as  well.  Comments 
included: 


(1)  Depends  on  the  displays 's  relevance  to  the  particular 
parameters  and  their  relationships  to  the  process. 

(2)  Choice  of  data  and  display  method,  e.g.  surface  vs. 
shading. 

(3)  Field  of  view  also  very  important. 

(4)  Data  base  management. 

(5)  Accuracy  is  another  feature. 

(6)  The  features  that  are  most  important  to  me  are  those 
that  are  required  by  the  user,  and  they  are  TBD. 

(7)  Visual  quality  (resolution),  field  of  view. 

(8)  Most  important  is  a and  b with  high  scene  complexity. 

(9)  In  my  opinion,  visual  quality  is  not  all  that  important 
for  engineering  application. 

(10)  “Refresh  speed”  should  be  broken  into  update  rate  and 
throughput  delay.  Need  >15Hz  and  <.l  sec.,  resp. 

(11)  Stereo  [5],  Sound  [6],  Force  feedback  [7] 

(12)  Scale  is  of  significant  importance.  Will  this  system 
provide  sufficient  information? 

(13)  The  choices  do  not  make  sense,  e.g.  graphic 
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presentations  are  tactile  (?) . Resolution,  latency, 
registration,  e.g.  are  important,  but  application  specific. 

(14)  Ease  of  access  and  use. 

(15)  Flexibility  of  viewpoint  (interactivity — but  not 
necessarily  tactile) . 

(16)  Information  lookup  & access 

14.  What  do  you  think  of  the  likelihood  that  information 
structures  and  VR  user  interfaces,  customized  to  support  specific 
users  will  be  useful  and  valuable  in  manufacturing: 


2 1 

/ 

P=  0 .2 

Low 

Probability 


4 7 

/ 

.4  .6 

Moderate 
Probability 


15 


.8  1.0 

High  Certain 

Probability 


n Mean Sta.  Dev. Range 

29  0.71  0.22  0-1 


Respondents  assigned  a high  probability  (expected  value  of 
0,71)  to  the  prospective  utility  in  manufacturing  of  information 
and  VR  interfaces,  as  illustrated  in  Figure  12,  Some  have 
caveats  to  the  optimism,  as  indicated  in  the  comments: 

(1)  Only  true  if  question  12  is  met. 

(2)  Pushes  development  down  the  learning  curve  more  rapidly. 

(3)  Manufacturers,  in  general,  cannot  afford  associated  cost 
for  unique  approaches. 

(4)  VR  is  simply  the  natural  evolution  of  interfaces  towards 
using  human  capabilities. 

(5)  It  is  just  a matter  of  time. 

(6)  PC's  are  not  yet  commonplace  in  manufacturing. 

(7)  Almost  certain  in  some  kinds  of  manufacturing,  e.g., 
where  collaboration  and  experimental  prototyping  are  important. 

(8)  This  is  a vague  question.  It  depends  upon  what  kind  of 
manufacturing . 

(9)  Not  clear  that  technology  will  mature  to  point  of 
usefulness. 

15.  What  would  be  the  value  added  by  customized  information 
presentations  on  VR  computer-interface  technology  for 
manufacturing?  (Please  indicate  with  an  **x”  on  the  scale) : 

1 1 5 7 12 

/ / / / / / 

0 1 2 3 4 5 

None  Moderate  High 

n Mean Sta.  Dev. Range 

26  3.52  1.04  0-5 
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The  respondents  are  optimistic  about  high  value  added,  as 
shown  in  Figure  13,  with  again  some  caveats  expressed  in  their 
comments : 

(1)  It  depends,  e.g.  for  inspection  it  is  high,  for  handling 
it  is  low  to  moderate. 

(2)  Increased  ability  to  model  processes. 

(3)  Customization  will  improve  utilization  of  system. 
Customization  of  CAD  systems  have  demonstrated  this. 

(4)  Again,  to  be  determined  by  evaluation  of  the  interface 
in  comparison  to  conventional  manufacturing  procedures. 

(5)  Always  concerned  with  maintenance  of  specialized 
software/hardware . 

(6)  Potential  is  high. 

(7)  Hard  to  estimate. 

(8)  Depends  on  the  application. 

(9)  At  the  moment,  it  is  low. 

(10)  The  ability  to  provide  appropriate  information  is  very 
relevant. 

(11)  Rapid  low-cost  experimental  prototyping;  training 
without  real  world  hazards;  perceptualization  of  complex 
information. 

(12)  Again,  it  depends  on  the  application. 

(13)  High  value  in  distributed  and  reconf igurable  (i.e.  JIT) 
applications. 

16.  Who  will  drive  information  presentations  and  VR  computer- 
interface  technology  for  manufacturing?  (Check  one) 

(a)  Developers  [ 7 ] (b)  Users  [ 6 ] (c)  Both  [ 16  ] 

n=29 


A majority  of  respondents  believe  that  both  developers  and 
users  will  drive  the  technology,  as  depicted  in  Figure  14, 
although  the  remainder  are  almost  evenly  split  between  users 
versus  developers  as  technology  drivers.  Comments  include: 

(1)  Those  that  require  quick  acceptance. 

(2)  Users  drive  the  need  and  ultimately,  the  success. 
Developers  drive  level  of  available  technology.  If  users  don't 
buy,  it's  dead.  If  developers  don't  provide  technology  to  meet 
users'  needs,  users  don't  buy,  and  it's  dead. 

(3)  A widely  used  application  is  used  by  VR  industry  to 
drive  down  costs  and  increased  developers/researchers. 

(4)  Developers  cannot  create  for  a merchant  without 
involving  the  customers  (at  least  not  if  they  expect  to  be 
successful  financially! ) . 

(5)  I hope! 

(6)  Developers  need  to  find  proponents  in  the  user  community 
who  can  effectively  sell  these  new  ideas. 

(7)  Purely  technology  looking  for  application! 
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(8)  Currently  the  feedback  loop  is  critical  to  the  full 
development  of  VR. 

(9)  It  is  unfortunate  that  developers  will  be  the  drivers, 
but  it  too  often  does  seem  to  be  the  case  with  new  technology. 

(10)  Developers  are  always  the  drivers. 

(11)  High  cost  applications  and  low-tech  users  =>  developers 
are  driver. 

17.  Who  will  be  primary  funding  sources  for  information 
presentations  with  VR  computer- interface  technology  for 
manufacturing?  (Check  one) 

(a)  Developers  [ 6 ] (b)  Users  [ 9 ] (c)  Government  [ 12  ] 

n=27 


As  shown  in  Figure  15,  a plurality  chooses  the  government  as 
the  primary  funding  source^  with  a substantial  number  picking  the 
users.  Few  think  that  the  developers  will  pick  up  the  tab. 
Comments  include: 

(1)  Government  funding  is  needed  for  "proof  of  concept.” 

(2)  Unless  a widely  used  application  is  selected  by 
government,  user  (industry)  will  have  to  provide  funding. 

(3)  Government  will  fund  at  least  until  there  is  a cost 
incentive. 

(4)  The  money  will  come  from  sale  of  commercial  products — 
government  (i.e.  DoD)  isn't  enough. 

(5)  Government  will  fund  only  initially — when  potential  is 
seen  users  and  developers  will  almost  certainly  drive 
development. 

(6)  Government  will  fund  until  several  good  examples  are 
fielded  and  widely  known. 

(7)  Realistically,  government  funding  is  probably  needed  as 
lead  funding. 

(8)  Government  and  developers  have  clear  important  roles  too 
(even  though  users  will  be  the  primary  funding  source) . 

(9)  Unfortunately,  organizations,  especially  manufacturing 
organizations,  do  not  have  the  capital  for  such  R&D. 

(10)  Government  will  fund  at  first — but  then  also  industry's 
(user's)  research  arms. 

(11)  Vendors  will  develop  hardware/ software,  manufacturers 
will  purchase  it.  Perhaps  government  will  fund  R&D. 

(12)  Venture  capital 

(13)  Unless  intellectual  property  rights  can  be  vested  with 
developers,  they  will  resist  Government  and  user  funding. 

18.  Which  technologies  do  you  believe  must  be  exploited  to  make 
information  presentations  with  VR  computer-interface  technology 
for  manufacturing  workable  and  affordable  to  users? 
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The  respondents  listed  a variety  of  technologies  in  their 
comments^  but  graphics  seems  to  be  important  to  many.  Some  say 
we  have  all  the  hardware  we  need,  that  software  is  the  limiting 
technology,  especially  intelligent  software: 

(1)  Communications  hardware  and  software. 

(2)  Crystallized  Eyeware  (head  mounted  devices) . 

(3)  Knowledge  Engineering,  Knowledge/Data/ Information  base 
work.  You  must  be  able  to  maintain  information  to  be  presented. 

(4)  Communications  and  high  speed,  high  resolution  graphics 

displays. 

(5)  Imaging/graphics.  Modular /object  software  good  display 
and  graphics  speed. 

(6)  HPCC,  force  feedback,  visual  displays,  audio. 

(7)  Improved  immersive /augmented  display  devices,  tracking 
devices,  tactile  feedback,  database  management,  parallel 
processing. 

(8)  Telecommunications  (fiber,  ATM,  ...);  graphics;  I/O 
architectures . 

(9)  Passive  sensors;  tactile-haptile  technology. 

(10)  Human  factors  technologies  to  ensure  that  the  users  get 
the  right  information  in  the  right  form  at  the  right  time. 

(11)  Hardware  (VLSI)  and  decreasing  size  w/  increasing 
portability. 

(12)  The  hardware  technologies  already  exist  for  VR  + 
information  presentations  (not  embedded  graphical  types  of  VR; 
but  text  based  with  simple  graphics) . So  software  + human 
factors  need  to  be  developed. 

(13)  Robust,  adequate  visual  displays;  multi-sensory 
displays  as  appropriate;  distributed,  low-cost  graphics  computing 

(14)  Graphic  computing  and  display  technologies. 

(15)  Sensing,  wireless  communications. 

(16)  Graphics  hardware,  network  software,  database  software, 
graphics  software. 

(17)  CAD  Display,  SMP,  Parallel  processing.  Digital  Video, 
Sensors,  HCI. 

(18)  Non-intrusive  position  sensing.  High  quality  see- 
through  displays. 

(19)  PC  based  VR,  RT  shop  floor  data  at  low-cost. 

(20)  High-resolution  light-weight  small  information 
displays;  software/hardware  capable  of  generally  large  scale 
environments;  appropriate  tracking  technology. 

(21)  Low-cost,  high-quality  displays;  intelligent, 
distributed  software;  multiple  modalities. 

(22)  In  general  (because  I don't  know  manufacturing) : high 
performance,  low  cost,  acoustical  rendering  engines;  wide  field 
of  view  for  high  resolution,  low  cost  HMDs;  improved  software 
packages  and  standards;  force,  tactile  feedback  device 
development. 

(23)  Human  factors  of  telerobotics  & flight  simulation. 

(24)  Much  smaller  & lighter,  & more  ergonomically  acceptable 
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displays;  get  away  from  "data  glove”  type  controls. 

(25)  High-bandwidth,  multi-media  networking,  highly 
distributed  software  technology. 

19.  Which  industries  or  types  of  processes  do  you  feel  can 
benefit  first  from  information  presentations  with  VR  computer- 
interface  technology  for  manufacturing  applications? 


The  respondents ' s suggestions  ranged  from  heavy  industry, 
aircraft  and  automotive  systems,  to  electronics  and  nanosystems: 

(1)  Discrete  parts  manufacturing  and  assembly,  maintenance, 
repair,  and  overhaul. 

(2)  Multi-media. 

(3)  Discrete  parts  manufacturing,  e.g.,  electronics, 
automobile,  airplane. 

(4)  Process  planning  in  a variety  of  industries. 

(5)  Diagnostics  and  maintenance  process. 

(6)  Precision  manufacturing,  e.g.  aircraft,  electronics, 

etc. 

(7)  Flexible  manufacturing  systems. 

(8)  Aerospace,  automotive,  entertainment,  information, 
telecommunication. 

(9)  Machined  parts;  CAD/CAM;  CAL;  Virtual  Co-location. 

(10)  Product  design,  process  design,  process  (monitoring 
and)  control. 

(11)  Processes  not  requiring  high  visual  detail,  e.g. , where 
untextured  polygons  are  sufficient  for  the  task  to  be 
accomplished. 

(12)  Any  requiring  dissemination  of  information  and 
communication  and  state  sharing  between  people. 

(13)  Custom  manufacturing  (small-runs,  complex  designs) ; 
hostile  materials  use,  "nano-manufacturing.” 

(14)  High  front-end  costs  in  tools/dies/assembly  facilities. 

(15)  Those  in  which  the  spatial  problems  of  part 
fabrication,  or  component  assembly,  are  dominant. 

(16)  Prototyping.  VR  is  useful  for  seeing  expensive 
mistakes  before  they  happen. 

(17)  Hi-tech,  CAD  based  systems — Automotive,  MF,  Heavy 
Equipment. 

(18)  All. 

(19)  Hazardous  environments — nuclear /toxic  with  robotic 
material  handling. 

(20)  Labor-intensive  work  requiring  templates. 

(21)  Design;  maintenance  training;  process  visualization. 

(22)  Those  involved  with  remote  handling  of  hazardous 
material;  complex  product  development  with  multiple  inputs  (such 
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as  aircraft  design) . 

(23)  Aerospace  & medical  equipment  manufacturing; 
electronics. 

(24)  High  through-put,  volatile  industries,  where 
flexibility  can  be  readily  translated  into  competitive 
advantages. 

20.  How  would  you  rank  the  necessary  growth  of  the  following 
technological  factors  for  the  integration  of  information 
presentations  on  VR  computer-interface  technology  into 
manufacturing?  (Please  indicate  with  an  ”x*'  on  the  scales  for 
each  factor) : 

0 3 8 6 10 

20a.  Computer  hardware?  / / / / / / 

0 1 2 3 4 5 

None  A Great  Deal 

11 Mean Sta.  Dev. Range 

27  3.29  1.13  0-5 

Improvement  of  computer  harware  was  deemed  very  important  by 
most  of  the  respondents,  as  depicted  in  Figure  16. 

0 2 8 7 10 

20b.  Communications  systems?  / / / / / / 

0 1 2 3 4 5 

None  A Great  Deal 

n Mean Sta.  Dev. Range 

27  3.33  1.02  0-5 


Improvement  of  communications  systems,  shown  in  Figure  17, 
was  deemed  slighly  more  important  than  improvement  in  computer 
hardware . 

0 1 2 7 17 

20c.  Software?  / / / / / / 

0 1 2 3 4 5 

None  A Great  Deal 

n Mean Sta.  Dev. Range 

27  3.91  0.90  0-5 

As  illustrated  in  Figure  16,  the  respondents  see  software 
improvement  as  more  important  than  improvement  in  hardware  or 
communications  systems. 
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0 1 1 15  10 

20d.  Human  Factors?  / / / / / / 

0 1 2 3 4 5 

None  A Great  Deal 

n Mean Sta.  Dev. Range 

27  3.73  0.74  0-5 

While,  on  the  average,  the  respondents  deem  enhancement  of 
human  factors  somewhat  less  important  than  improvement  in 
software  (although  more  improtant  than  improvement  in  hardware  or 
communication  systems) , there  is  more  agreement  as  to  its 
importance  (with  a smaller  standard  deviation  than  the  standard 
deviation  for  the  judgement  on  software) , as  can  be  seen  in 
Figure  19.  Comments  include  a need  for  standards: 

(1)  Standards,  standards,  standards. 

(2)  Not  a technology  issue.  Manufacturing  practices  and 
shop  floor  control  must  progress  more  than  any  of  these! 

(3)  All  of  these  must  be  developed  a great  deal  for  ideal  VR 
systems,  but  this  does  not  necessarily  mean  that  the  technology 
could  not  be  useful  in  its  current  state  for  some  manufacturing 
applications . 

21.  How  well  can  standards,  open  architectures,  and 
interoperability  be  achieved  in  information  presentations  with  VR 
computer-interface  technology  to  define  a framework  across 
multiple  vendors  of  hardware  and  software  and  users?  (Please 
indicate  with  an  ”x”  on  the  scale) : 

1 7 3 7 7 

/ / / / / / 

0 1 2 3 4 5 

Poor  Excellent 


n Mean Sta.  Dev. Range 

25  2.92  1.34  0-5 

The  respondents  vary  greatly  in  their  expectations,  although 
only  one  believes  the  opportunity  for  achieving  standards,  open 
architectures,  and  interoperability  is  very  poor.  The 
expectation  is  a bit  more  optimistic  than  dead  center  (2.9),  but 
the  standard  deviation  and  range  of  responses  shows  the  lack  of 
agreement  in  the  group,  as  graphically  displayed  in  Figure  20. 
Comments  include  the  view  of  many  that  it  is  premature  to  worry 
about  standards,  even  though  they  will  be  needed  evntually: 

(1)  I believe  it  is  too  early  in  the  development  of  this 
technology  to  answer  this  question. 

(2)  Open  software  will  be  hard,  hardware  will  be  easier. 
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(3)  Similar  challenges  as  in  other  types  of  applications. 

(4)  Unfortunately,  with  VR  really  in  its  infancy,  standards 
could  slow  growth  but  will  be  needed. 

(5)  This  is  a challenge  - there  is  no  question. 

(6)  It  has  been  done  in  other  domains;  why  not  here? 

(7)  Standards  and  agreed  upon  architectures  will  be  needed 
to  gain  critical  mass  of  users. 

(8)  Tough  call — standards  here  are  a must  for  VR  interfaces 
to  succeed. 

(9)  This  is  an  unknown.  Different  approaches  must  be  tried 
before  a standard  can  be  developed. 

(10)  They  can  be,  but  there  is  only  modest  movement  in  the 
fragmented  VR  industry  and  research  commonly  toward  this  goal. 

(11)  Too  early  in  the  game  for  standards. 

(12)  It  is  hard  for  me  personally,  to  envision  such  a broad 
scope  of  application. 

(13)  A great  deal  can  be  done  on  standardizing  file  formats, 
Communication  protocols,  etc.  Dynamics  are  harder. 

(14)  VR  is  not  fundamentally  different  from  any  other 
technology. 

(15)  It  is  much  too  early  to  standardize! 

(16)  Experience  shows  that  "defacto”  standards  are  usually 
accepted. 

(17)  They  have  been  very  poorly  achieved  to  date,  and  may  be 
premature  before  the  technology  has  further  matured. 

(18)  They  could  if  they  existed,  but  it's  premature. 

(19)  Field  has  to  shake  out  first. 

(20)  Can't  be  developed  in  a vacuum — need  some  technology 
base  before  meaningful  guidelines  can  be  developed. 

22.  Does  special  equipment  now  required  for  realism  - 
headmounted  displays,  data  gloves  - reduce  or  enhance  the  ability 
of  the  user?  (Please  indicate  with  an  ”x"  on  the  scale) : 

2 5 5 7 6 

/ / / / / / 

0 1 2 3 4 5 

Reduce  Enhance 

ri Mean Sta.  Dev. Range 

25  2.80  1.44  0-5 

The  respondents  views  are  quite  scattered^  with  no 
agreement^  as  shown  in  Figure  21,  but  on  the  average  they  lean 
toward  the  view  (with  a mean  of  2,8)  that  the  special  equipment 
enhances  the  ability  of  the  user.  Comments  include  views  on  the 
advantages  and  disadvantages  of  the  special  equipment : 

(1)  It  is  application-dependent. 

(2)  Highly  dependent  on  the  particular  application  and 
quality  of  the  display. 
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(3)  Enhances  quality,  reduces  comfort. 

(4)  Enhances  focus,  reduces  mobility. 

(5)  A lot  of  devices,  today  are  crude,  but  with  significant 
improvements,  one  can  see  the  potential. 

(6)  No  questionl  But  costs  and  biological  as  well  as 
psycho-social  reactions  to  the  systems  will  play  a role. 

(7)  An  empirical  question  that  probably  gives  a differed 
answer  for  each  task. 

(8)  We  believe  they  enhance,  but  we  have  little  quantitative 
evidence. 

(9)  Getting  used  to  a new  technology  presents  an  initial 
cultural  shock,  which  once  overcome  will  prove  valuable. 

(10)  This  depends  very  much  on  the  task  being  performed. 

(11)  Hard  to  say  since  I've  seen  both  enhancement  and 
reduction  depending  on  application  & technology  used. 

(12)  "Ability  of  the  user" — to  do  what? 

(13)  Datagloves  are  pointless  without  tactile/ force 
feedback.  HMD's  are  probably  on  the  way  out  with  the  HITL  VRD. 

(14)  Not  the  manufacturing  roadblock. 

(15)  What  other  choices  are  there? 

(16)  Depends  heavily  on  application. 

(17)  Depends  on  what  you  mean  by  "the  ability  of  the  user.” 
Enhances  some  abilities,  reduces  others. 

(18)  It  enhances  cognitive/perceptual  ability  but  with 
present  technology  at  some  cost  in  cumbersomeness  and  "simulation 
sickness . ” 

(19)  There  needs  to  be  lots  of  improvement  here,  as  I've 
noted  before. 

(20)  Cumbersome  with  limited  capability. 

23.  How  well  can  VR  tools  for  the  information  presentation  with 
VR  computer-interface  technology  be  integrated  with  manufacturing 
technology  tools?  (Please  indicate  with  an  ”x"  on  the  scale) : 

2 17  4 9 

/ / / / / / 

0 1 2 3 4 5 

Poor  Very  Well 

n Mean  Sta.  Dev. Range 

23  3.10  1.26  0-5 

Most  believe  that  VR  tools  and  interface  technology  can  be 
integrated  at  least  reasonably  well  with  manufacturing  technology 
(with  a majority  seeing  integration  going  quite  well),  as  shown 
in  Figure  22.  Comments  include  the  view  that  it  is  already  done, 
or  that  it  will  be  difficult  to  do: 

(1)  I am  involved  in  such  a project  involving  manufacturing 
cell  layout  flow  path  design. 

(2)  Currently? 
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(3)  The  information  display/operator-interface  systems  must 
be  integrated  with  manufacturing  tools. 

(4)  Augmented  reality  (superimpose  graphics  on  real  world) 
is  promising. 

(5)  Deneb  Robotics  has  demonstrated  this. 

(6)  Very  well,  but  it  will  take  an  effort  (i.e.  money, 
support,  R&D) . 

(7)  In  many  cases,  they  overlap.  CAD-CAM  is  certainly  on 
the  VR  dimension. 

(8)  We  expect  that  VR  is  like  color — when  appropriate  all 
will  wonder  how  we  did  without  - but  it  can  always  be  misapplied. 

(9)  I suspect  some  changes  in  manufacturing  practices  are 
needed  to  fully  integrate  VR. 

(10)  Where  there's  a will,  there's  a way. 

(11)  Depends  on  what  manufacturing  technology  tools  you're 
interested  in. 

(12)  In  my  opinion,  hard  to  predict. 

(13)  Supporting  infrastructure  cannot  handle  VR  demands. 

(14)  Augmented  reality  provides  a direct  mechanism  for 
integration. 

(15)  This  will  be  more  of  a problem  than  many  believe  - I 
think  the  control  aspects  are  most  problematic. 

(16)  In  theory-great;  in  practice  there  is  no  existing 
software  or  communications  infrastructure  in  manufacturing. 

24.  How  would  you  rate  the  applicability  of  the  fidelity  of 
information  presentation  achieved  with  VR  computer-interface 
technology  in  manufacturing  technology  for  discrete  part 
production  in  each  of  the  following  areas?  (Please  indicate  with 
an  *'x'*  on  the  scales)  : 

0 1 3 2 18 

24a.  Training?  / / / / / / 

0 1 2 3 4 5 

Poor  Excellent 


n Mean Sta.  Dev. Range 

24  3.93  0.94  0-5 

The  response  was  overwhelming  that  fidelity  is  applicable  to 
training^  as  shown  in  Figure  23 . 


24b.  Product  design? 
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The  respondents  also  think  that  fidelity  is  very  applicable 
to  product  design,  but  somewhat  less  so  than  for  training,  as 
depicted  in  Figure  24 . 


3 

4 

2 

5 11 

24c.  Process  design? 

/ — 

-/ — 

-/ — 

-/- 

— / / 
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3 

4 5 

Poor 

Excellent 

n Mean  Sta.  Dev. 

Range 

26  3.10  1.45 

0-5 

The  views  on  process  design  are  more  diverse 

than  for 

product  design,  as  shown  in  Figure 

25, 

with  fidelity  being 

seen 

as  somewhat  less  applicable . 
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11 

24d.  Process  validation?  / 

— / /■ 

-/ — 

-/ 

0 

1 

2 3 

4 

5 

Poor 

Excellent 

n Mean  Sta.  Dev. 

Range 

24  3.28  1.33 

0-5 

Fidelity  is  believed  to  be  a . 

bit 

more  important 

for  process 

validation  than  for  process  design 

, as 

graphed  in 

Figure  26 

, but 

there  is  a diversity  of  views  (as 

evidenced  by  a ; 

standard 

deviation  of  1.3). 
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4 5 
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24e.  Enterprise  integration?  / 

— / / 

--/ — 

-/ 

0 

1 

2 3 

4 

5 

Poor  Excellent 

n Mean Sta.  Dev. Range 

23  2.88  1.54  0-5 

Fidelity  is  not  seen  as  highly  applicable  to  enterprise 
integration,  as  shown  in  Figure  27.  Comments  for  the 
applicability  of  fidelity  to  the  various  functions  include: 

(1)  Validation  requires  simulation  modes  well  beyond  current 

VR. 

(2)  Product  design  harder  because  higher  visual  fidelity 
important. 

(4)  Fidelity  is  not  always  necessary  or  desirable. 

(5)  I am  a little  uncertain  of  the  meaning/ intent  of  this 
question. 
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(6)  In  process  design,  mostly  due  to  poor  models;  in 
enterprise  integration,  graphic  representations  are  quite  poor. 

25.  Can  the  US  manufacturing  industry  afford  to  apply  high- 
fidelity,  customized  information  presentations  with  VR  computer- 
interface  technology?  (Please  indicate  with  an  ”x"  on  the 
scale)  : 


1 5 4 4 6 

/ / / / ! / 

0 1 2 3 4 5 

Can  not  Can 

n Mean Sta.  Dev. Range 

20  2.84  1.30  0-5 

The  views  on  affordability  are  all  over  the  scale,  but  the 
group  leans  toward  the  view  (with  a mean  of  2.84)  that  the 
manufacturing  industry  can  afford  to  apply  high-fidelity, 
customized  information  presentations  with  VR  computer  interface 
technology,  as  shown  in  Figure  28.  Comments  also  vary  across  the 
spectrum: 

(1)  If  competitors  can  afford  it,  can  they  afford  not  to? 

(2)  The  industry  cannot  afford  not  to  apply  high  quality 
display  techniques. 

(3)  Can  afford  it,  but  with  limited  "customization.” 

(4)  Cost  of  not  doing  so  is  expensive,  downstream  rework. 

(5)  At  this  time  the  benefits  have  not  been  clearly 
demonstrated  to  industry. 

(6)  How  can  it  NOT  from  a global  competitive  perspective. 

(7)  Certain  industries  will  gain  some  advantage  irrespective 
of  cost  to  use  VR. 

(8)  It  cannot  afford  not  to  for  the  future. 

(9)  In  specific  cases  it  can  afford  it. 

(10)  Can't  afford  PC's  yet. 

(11)  It  can  if  the  parts  costs  drop. 

(12)  Things  are  getting  rapidly  cheaper. 

(13)  Do  not  know  enough  about  economics  of  manufacturing, 
though  I'd  be  surprised  if  the  answer  were  "cannot." 

(15)  Depends  on  the  industry. 

(16)  Not  now  - applicable  mostly  to  "niche"  markets  - need 
subsidies  to  be  developed  to  point  of  viability. 

(17)  Not  with  today's  technology — need  5-10-fold  decrease  in 
costs . 

26.  How  much  will  cost  of  equipment  to  support  a high  quality 
graphical  and  interaction  interfaces  (future  generations  of 
workstations,  head-mounted  displays,  and  data  gloves)  effect 
acceptance  of  a virtual  workplace?  (Please  indicate  with  an  "x" 
on  the  scale) : 
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0 3 4 5 12 

/ / / ! / / 

0 1 2 3 4 5 

None  A Great  Deal 

n Mean Sta.  Dev. Range 

24  3.62  1.06  0-5 

Cost  is  deemed  a very  Important  influence  on  acceptance  of 
the  virtual  workplace,  as  shown  in  Figure  29,  However,  as  the 
comments  indicate,  benefits  must  also  be  part  of  the  equation: 

(1)  It  should  really  be  a cost  versus  benefit  question. 

(2)  This  is  so  strongly  dependent  on  the  particular 
application  that  I can't  give  one  rating  value. 

(3)  HMD's  will  live  or  die  by  price. 

(4)  If  benefits  are  clear,  price  will  not  be  a significant 
barrier  in  the  long  run. 

(5)  The  bottom  line  for  Industry  is  cost  versus  benefits. 

(6)  Cost  must  go  down  and  revenue  incentive  must  go  up. 

(7)  Extremely  important. 

(8)  High  quality,  head-mounted  displays  are  not  necessary. 

(9)  Major  driver  for  all  international  manufacturing 
companies . 

(10)  Price  is  very  important  - when  desktop  PC  dropped  into 
the  $10K  range  they  become  available. 

(11)  I think  changes  in  cost  without  improvement  in  quality 
might  have  little  effect. 

(12)  Depends  on  benefits. 

(13)  The  question  is  not  cost.  It  is  added  value!  Usually 
you  can't  introduce  equipment  like  this  alone  in  one  part  of  an 
enterprise. 

(14)  American  industry  is  resistant  to  high-capital 
technologies  - if  not  then  affordable,  VR  will  be  bypassed. 

27.  Do  you  know  of  direct  requirements  for  VR  interface 
presentations  for  any  manufacturing  systems?  (Please  check  one) 

(a)  Yes  [ 5 ] (b)  No  [ 11  ] (c)  Maybe  [ 8 ] 

n=24 


Only  21%  of  the  respondents  know  of  a direct  requirement  for 
certain,  as  shown  in  Figure  30,  Nearly  half  are  certain  they  do 
not  know  of  a requirement.  Comments  include: 

(1)  For  manufacturing  cell  layout  and  material  flow  systems. 

(2)  For  VR  specifically  - NO.  For  user  interface 
presentation  in  general  - YES. 

(3)  Ease  of  use  and  setup. 

(4)  Deneb  Robotics. 

(5)  The  WVHTC  Foundation  "Virtual  Company  Distributed 
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Manufacturing  Project." 

(6)  I don't  know  if  you'd  call  them  "requirements,"  but 
there  are  standards,  etc.,  in  for  example  the  OSI  ISO  Reference 
Model.  Standards  for  "open"  systems — interoperability. 

(7)  Maintenance/accessibility  of  equipment;  facility  design/ 
operation. 

(8)  To  a limited  degree. 

(9)  Textile  industry. 

(10)  Boeing  777.  Several  aspects  of  VR  technology  were 
useful  in  design  process. 

28.  Will  information  presentation  with  VR  computer-interface 
technology  have  an  overall  worth  in  integrating  companies, 
enterprises,  or  industries  more  intelligently  into  the 
manufacturing  process?  (Check  one) 

(a)  Yes  [ 20  ] (b)  No  [ 0 ] (c)  Maybe  [ 7 ] 

n=27 

Not  one  respondent  is  certain  that  there  is  no  integrating 
utility  of  VR  technology,  as  graphed  in  Figure  31.  Comments 
include: 

(1)  Don't  understand  what  (who?)  is  being  integrated: 
manufacturing  companies  in  manufacturing? 

(2)  Data  visualization  promising  for  financial  flow. 

(3)  Absolutely! 

(4)  Absolutely.  The  ability  to  convey  enormous  amounts  of 
information  in  a VR  environment  will  allow  people  and 
surroundings  to  communicate  and  interact  in  ways  which  are  much 
more  natural  and  certainly  more  flexible  than  text,  databases, 
and  drawings. 

(5)  Potential  is  high  but  .. 

(6)  Absolutely,  they  present  the  potential  for  serious  cycle 
time  reduction. 

29.  Will  information  presentation  with  VR  computer-interface 
technology  integrate  vendors/ suppliers  to  a greater  extent  into 
the  manufacturing  process?  (Please  check  one) 

(a)  Yes  [ 21  ] (b)  No  [ 0 ] (c)  Maybe  [ 6 ] 

n=27 


Again,  the  respondents  are  optimistic  about  the  utility  for 
achieving  integration  as  a result  of  using  the  technology,  as 
shown  in  Figure  32.  Comments  include: 

(1)  If  vendors /suppliers  provide  "components  for  VR" 
engineering  part  models  for  evaluation. 

(2)  Manufacturing  has  to  improve  integration  with  vendor/ 
supplier  to  be  competitive. 
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(3)  Certainly  expect  it  will. 

(4)  Potential  is  high  but  ... 

(5)  Good  question,  and  my  answer  is  a firm  yes. 

(6)  VR  does  not  solve  any  integration  problems. 

(7)  Mainly  because  everything  will  be  in  the  digital  domain 
making  it  more  probable. 

(8)  They'd  better,  if  they  are  worth  the  effort! 

(9)  Only  if  systems  are  OPEN,  proprietary  technologies  will 
create  isolation,  not  integration. 

30.  Please  make  any  additional  comments  below: 

(1)  I found  this  questionnaire  difficult  to  respond  to  for 
the  following  reasons: 

(a)  All  computer  based  systems  (which  is  essentially 
everything  today,  from  cars  and  appliances  to  large 
manufacturing  systems)  have  an  absolute  requirement  for 
information  presentation  to  various  levels  of  users. 
Therefore,  all  of  these  systems  must  have  good  displays 
and  VR  technologies  are  an  evolutionary  step  in  this 
area. 

(b)  The  quality  of  displays  relative  to  this  information 
being  presented  and  the  operator's  needs  is  critical 
but  not  well  developed  or  understood  at  the  present 
time.  As  a result,  a **bad”  display  is  worse  than  no 
display  at  times  and  a lot  of  VR  displays  are  still  at 
the  "bad"  stage,  so  a judgement  on  them  is  hard  to 
make. 

(2)  Sorry,  must  miss  NIST  workshop. 

(3)  VR  both  text  and  graphics  based  will  play  a significant 
role  in  the  development  of  the  WVHTC  Foundation's  "Virtual 
Company  Model”  and  most  importantly  with  respect  to  it  efforts 
with  distributed  manufacturing. 

(4)  I think  it's  very  valuable  to  have  a workshop  on  these 
issues  now.  We  have  made  some  gains  in  "VR  in  training,”  and  we 
should  see  if  the  same  principles  apply  in  "VR  in  manufacturing." 
By  opening  up  a "second  front”  so  to  speak,  we  will  make  faster 
progress  to  attain  the  benefits  that  VR  interfaces  afford. 

(5)  While  high  quality  graphical  displays  + head  mounted 
displays  are  nice,  I think  that  a lower  tech  solution 
(workstation  quality  displays,  Pentium  class  machines)  would 
provide  adequate  solutions.  Also  a missing  component  of  many  VR 
systems  is  the  people.  Once  inside  a VR  world,  a person  still 
gets  a great  deal  of  benefit  by  sharing  information  with  others. 

(6)  As  I've  indicated,  I have  so  little  knowledge  of 
manufacturing  that,  in  spite  of  my  VR  and  human  factors 
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expertise,  I cannot  give  useful  answers  to  many  of  these 
questions  without  having  more  information  about  processes  and 
challenges  in  manufacturing. 

(7)  My  philosophy  is  that  VR  is  not  a single  "thing"  or 
"entity,"  but  consists  of  a series  of  "features"  (high  res. 
graphics,  stereo,  interactivity,  data  glove,  etc.)  any  of  which 
can  be  present  or  absent.  Some  of  these  features  matter  more 
than  others  for  cost  effectiveness  of  this  technology  to  any 
particular  application  area. 
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DEVELOPER  USER  RESEARCHER 
TYPE  OF  EXPERIENCE 


Figure  1 
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Figure  2 
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Figure  4 
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CURRENT  LEVEL  OF  DEVELOPMENT  IN 
HUMAN  FACTORS  IN  COMPUTER  INTERFACE 
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Figure  5 
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Figure  6 
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RESPONDENT’S  FAMIUARITY 
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Figure  7 
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INITIAL  TEXT-BASED  VR  PRESENTATIONS 
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Figure  8 
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RESPONDENT’S  FAMIUARITY  WITH 
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Figure  9 
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INITIAL  GRAPHICS-BASED  VR 
PRESENTATIONS  IN  MANUFACTURING 
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Figure  10 


NUMBER  OF  RESPONDENTS 


FINAL  REPORT 


44 


PRIORITIZED  FEATURES 
IN  VR  PRESENTATION 
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Figure  11 
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Figure  12 
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VALUE  ADDED  WITH  CUSTOMIZED 
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Figure  13 
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DRIVERS  OF  INFORMATION  AND 
INTERFACES  FOR  MANUFACTURING 
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Figure  14 
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FUNDING  SOURCES  FOR  VR  INFORMATION 
AND  INTERFACES  FOR  MANUFACTURING 
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Figure  16 
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Figure  17 
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Figure  18 
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Figure  19 
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Figure  20 
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IMPACT  OF  SPECIAL  EQUIPMENT 
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Figure  21 
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Figure  22 
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Figure  23 
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APPLICABIUTY  OF  PRESENTATION  FIDELITY 
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Figure  24 
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APPUCABIUTY  OF  PRESENTATION  FIDELITY 
IN  VR-BASED  PROCESS  DESIGN 
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Figure  25 
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APPLICABIUTY  OF  PRESENTATION  FIDELITY 
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Figure  26 
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APPUCABIUTY  OF  PRESENTATION  FIDELITY 
IN  VR-BASED  ENTERPRISE  INTEGRATION 
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AFFORDING  CUSTOMIZED  INFORMATION 
PRESENTATION  IN  VR  INTERFACES 
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Figure  28 
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EFFECT  OF  EQUIPMENT  COST  ON 
ACCEPTANCE  OF  A VIRTUAL  WORKPLACE 
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Figure  29 
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KNOWN  REQUIREMENTS  FOR  VR 
PRESENTATIONS  FOR  MANUFACTURING 
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Figure  31 
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WORTH  OF  VR-PRESENTATION  IN 
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Figure  32 
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BACKGROUND  INFORMATION: 

APPLICATION  OF  INFORMATION  AND  INTERFACE  TECHNOLOGY 
FOR  VIRTUAL  MID  DISTRIBUTED  MANUFACTURING  OF  DISCRETE  PARTS 


PURPOSE 

The  National  Institute  of  Standards  and  Technology  (NIST)  is 
supporting  new  initiatives  in  manufacturing  technology  which 
promise  to  have  a major  impact  on  the  national  economy.  The 
Intelligent  Systems  Division  (ISD)  of  NIST  is  exploring  the 
potential  applications  of  human  information  and  interface 
technology  for  virtual  and  distributed  manufacturing.  The 
purpose  of  the  enclosed  enquiry  is  to  determine  some  of  the 
issues  in  human  factors  and  virtual  reality  (VR)  for 
manufacturing  from  the  user's  perspective  and  to  build  a new 
vision  for  manufacturing  systems. 

Key  questions  relate  to  the  attributes  of  human  information 
interactions  through  10  interfaces  and  their  impact  on  potential 
leveraging  of  manufacturing  technology.  Who  needs  to  look  at 
information?  What  kinds  of  information  are  appropriate  for  each 
user  to  support  real-time,  collaborative  decision-making?  What 
differences  are  there  in  information  needed  for  different  users  - 
individually  or  in  new  collaborative  process  within  a "virtual” 
presentation  environment?  What  are  the  important  issues  in  how 
users  choose  to  view  information  - both  text  or  graphics?  What 
human  factors  issues  and  design  considerations  should  be  included 
in  a VR  computer-interface?  How  can  the  information  content  and 
presentation  best  be  approached  with  VR  computer-interface 
technologies? 

NEW  INITIATIVES 

Manufacturing  technology  is  currently  a focus  of  national 
initiatives  sponsored  by  numerous  government  organizations  and 
industrial  consortia.  There  is  a commitment  of  funding  by 
programs  in  the  Department  of  Commerce  (Advanced  Technology 
Program  [ATP]),  Department  of  Defense  (Technology  Reinvestment 
Program  [TRP]),  as  well  as  the  Department  of  Energy,  NASA,  and 
other  Federal  and  state  agencies.  All  of  these  programs  address 
the  necessity  of  boosting  US  manufacturing  capabilities. 

In  this  regard  NIST  has  been  given  an  expanded  role  in 
standardizing  technological  infrastructures  and  applications  in 
such  programs  as: 

* Manufacturing  Research  Technology  Centers  (METCs) , which  have 
become  major  regional  technology  resources 

* The  Automated  Manufacturing  Research  Facility  (AMRF) , an  open 
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architecture  test-bed  for  the  automated  manufacture  of  discrete 
parts 

* The  PDES-STEP  (Product  Data  Exchange  using  STEP  - Standard  for 
the  Exchange  of  Product  Model  Data)  Program  for  standardizing 
manufacturing  data  representations. 

Virtual  Reality 

Virtual  reality  is  rapidly  becoming  technically  and 
economically  feasible.  VR  has  many  definitions  and  formats.  At 
the  ambitious  end  of  the  spectrum,  VR  allows  a user  to  have  the 
illusion  that  he  is  immersed  in  a synthetic  or  remote 
environment.  He  experiences  sensory  information,  such  as  three- 
dimensional  vision  and  binaural  sound,  that  is  generated  by  a 
computer  or  originates  from  remote  sensors  (or  is  a combination 
of  the  two  sources) . He  interacts  kinesthetically  with  computer 
generated  or  teleoperated  objects.  At  the  less  ambitious  end  of 
the  spectrum,  VR  allows  a user  to  operate  in  a text-based 
interactive  modes  (i.e.,  on-line  dungeons  and  dragons)  or  a 
graphics-based  mode  to  manipulate  objects,  or  interact  with  an 
environment,  as  displayed  in  two  dimensions  on  a monitor 
(computer  or  workstation)  or  on  a head-mounted-display.  For  the 
purposes  of  manufacturing  applications,  the  more  modest  version 
of  VR  is  acceptable  along  with  the  more  ambitious  version. 

ISD  Initiative  in  Information  and  Interfaces  for  Manufacturing 

The  Intelligent  Systems  Division  (ISD)  and  the  Factory 
Automation  Systems  Division  (FASD)  of  the  Manufacturing 
Engineering  Laboratory  (MEL)  at  NIST  are  concerned  with 
standardizing  information  protocols  and  interface  technologies 
and  techniques  for  manufacturing  technology.  ISD  is  examining 
the  human  factors  involved  in  interfaces  for  appropriate 
information  interactions.  NIST  is  interested  in  exploring 
virtual  and  distributed  manufacturing  with  initial  testbed 
efforts  in  the  production  of  discrete  parts.  Primary  target 
areas  include:  Engineering  Design,  Product  Planning,  Process 
Planning,  Factory  Simulation,  Monitoring  and  Control. 

ISD  is  working  with  users  and  developers  to  coordinate 
standard  information  protocols  and  interfaces,  which  will 
eventually  be  reflected  in  the  integration  of  virtual  reality  in 
a manufacturing  framework.  This  human  factors  initiative  will 
have  a major  impact  in:  information  content  for  each  user; 
presentation  bandwidth  and  fidelity;  presentation  modality  - real 
time  or  non-real  time  and  mixes  of  text,  graphics,  pictures, 
videos,  and  other  sensory  modalities. 


VR  Threads 
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Virtual  reality  may  be  able  to  integrate  intelligent 
manufacturing  systems:  products,  processes,  and  enterprises.  The 
potential  impact  of  virtual  reality  may  be  dramatic.  VR,  which 
will  be  defining  itself  from  the  user  pull  and  the  technology 
push  over  the  next  several  years,  has  several  common  threads: 

* Virtual  reality  provides  new  worlds  for  one  or  more  users. 
Related  terminology  is:  artificial  worlds,  virtual 

environments,  and  synthetic  environments. 

* Some  VR  systems  attempt  to  give  the  user  a sense  of 
telepresence  by  immersive  sensation  (vision,  sound,  tactile 
(pressure) , haptile  (force) . Virtual  reality  may  apply  to 
single  sites  with  human/machine  interfaces  to  co-located 
computers,  or  to  interconnected,  distributed  systems  at  distant 
sites  linked  through  communications  networks. 

* Virtual  reality  establishes  a framework  which  will  ultimately 
integrate  many  areas  of  technology:  human  factors,  computer 
hardware  and  software,  and  communications. 

* Virtual  reality  permits  each  user  to  participate  in  a 
collective  effort. 

Virtual  reality  is  supported  by  enabling  technology  programs 
in  the  following  areas: 

* Communications:  the  National  Information  Infrastructure  (Nil) 

* High  performance  computation:  High  Performance  Computing  and 
Communications  (HPCC) 

* Data  and  interface  standards:  PDES-STEP 

* Hardware  including  head  mounted  displays  (HMDs) , body  suits, 
exoskeletons,  data  gloves,  large  screens 

* Human  factors  studies. 

VR  to  date  has  been  focused  primarily  in  four  application 
areas : 

(1)  Simulation  - which  yields  new  views  and  perspectives  which 
are  unattainable  by  current  methods. 

(2)  Teleconferencing  participation  - for  "meeting  at  a distance,” 
with  a sense  of  telepresence,  in  a collaborative,  immersive 
environment  for  joint  activities,  including  discussions, 
interviews,  planning,  and  training 

(3)  Training  and  Education,  including: 
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* Integration  of  training  for  personnel  at  distributed  sites 
with  distributed  elements 

* Using  simulations  or  synthetic  environments  to  achieve  cost 
savings  by  not  using  "real  elements;  when  "real”  elements  are 
unavailable;  and  to  protect  personnel  in  training  for  hazardous 
operations  and  environments,  e.g.,  nuclear  environments,  toxic 
contamination,  or  live-fire  conditions 

* Students  using  educational  virtual  reality  toolsets  to  build 
their  own  virtual  worlds,  to  immerse  themselves,  and  to 
interact  in  "what  if?"  situations 

* Students  can  learn  by  interacting  with  unavailable  and  out-of- 
size  components  and  environments,  such  as  observing  a real-time 
manufacturing  process  through  a magnifying  glass. 

(4)  Mutual  participation  in  shared  activities  in  a synthetic 
environment.  The  most  striking  current  participatory 
applications  of  virtual  reality  are  for;  Entertainment  (several 
theme  parks  have  simulator  rides  and  other  are  currently  being 
design  as  entire  virtual  reality  theme  parks) ; Military  (to 
test  new  technology  or  tactics,  and  to  train  personnel  at 
distributed  sites  in  synthetic  environments) . 
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ENQUIRY  FORM: 

APPLICATION  OF  INFORMATION  AND  INTERFACE  TECHNOLOGY 
FOR  VIRTUAL  AND  DISTRIBUTED  MANUFACTURING  OF  DISCRETE  PARTS 

PURPOSE 

The  purpose  of  this  questionnaire  is  to  determine  a vision 
for  human  factors  in  information  and  interfaces  needed  to 
integrate  virtual  reality  technologies  into  the  virtual  and 
distributed  manufacturing  of  discrete  parts.  You  are  asked  to 
address;  (1)  a definition  of  the  requisite  information  and 
interfaces  for  virtual  and  distributed  manufacturing 
applications,  (2)  human  factors  approaches  in  information  and 
interface  technologies,  and  (3)  leveraging  of  efforts  in  specific 
topical  areas.  The  results  of  this  questionnaire  will  allow  the 
Intelligent  Systems  Division  (ISD)  and  the  Manufacturing 
Engineering  Laboratory  (MEL)  of  the  National  Institute  of 
Standards  and  Technology  (NIST)  to  determine  needed  standards  and 
protocols  for  information  and  interfaces.  Robotic  Technology 
Inc.  is  working  on  this  questionnaire  under  contract  to  NIST. 

PROCEDURE 

You  have  been  provided  with  brief  background  information 
describing  human  factors  and  virtual  reality  for  manufacturing. 
Please  familiarize  yourself  with  this  material  before  answering 
the  questions  using  your  expert  judgment  and  opinion. 

Please  fax  only  this  completed  form  (not  the  background 
information),  as  soon  as  possible,  to  (301) -983-2509 . If  you  have 
any  questions  or  comments,  please  call:  Dr.  Joe  Iseman,  Robotic 
Technology  Inc.,  (301) -983-2465 . Thank  you  for  your 
participation  and  cooperation. 

YOUR  NAME 

Last  First 

POSITION  or  TITLE 

ORGANIZATION/ AGENCY 

ADDRESS 


PHONE 

FAX 


E-Mail 
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1.  With  respect  to  VR,  are  you  a potential:  ( Please  check  one) 
(a)  Developer  [ ] (b)  User  [ ] (c)  Researcher  [ ] 

Comments  (if  any) ; 


2.  How  much  do  you  know  about  virtual  reality?  (Please 
indicate  with  an  **x"  on  the  scale)  : 

/ / / / / / 

0 1 2 3 4 5 

Don't  Know  Much  Very  Knowledgeable 

Comments  (if  any) : 


3.  How  would  you  best  characterize  the  field  of  "Virtual 
Reality"?  (Please  indicate  with  an  "x"  on  the  scale) : 

/ / / / / / 

0 1 2 3 4 5 

Figment  of  Imagination  Realistic 

Comments  (if  any) : 


4.  How  much  do  you  know  about  human  factors/ergonomics  for 
defining  information  presentations  with  computer-interface 
technology  and  techniques?  (Please  indicate  with  an  "x"  on  the 
scale)  : 


/ / / / / / 

0 1 2 3 4 5 

Don't  Know  Much  Very  Knowledgeable 

Comments  (if  any): 
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5.  How  would  you  best  characterize  the  areas  of  human  factors/ 
ergonomics  for  defining  information  presentations  with  computer- 
interface  technology  and  techniques?  (Please  indicate  with  an 
”x**  on  the  scale)  : 

/ / / ! / ! 

0 1 2 3 4 5 

Inadequate  Well  Developed 

Comments  (if  any) : 


6.  Are  information  presentations  currently  adequately  embedded 
in  the  manufacturing  process?  (Check  one) 

(a)  Yes  [ ] (b)  No  [ ] (c)  Maybe  [ ] 

Comments  (if  any); 


7.  For  the  kinds  of  manufacturing  activities  with  which  you  are 
familiar,  what  kinds  of  information  would  typical  users  require 
to  perform  their  job  remotely:  (Please  comment  on  as  many 
activities  as  you  wish) 

7a.  Shop  floor  supervisors  


7b.  Machine  operators 


7c.  Inspectors 


7d.  Material  handlers 


7e.  Maintenance  and  repair  personnel 


7f.  Supply  room  personnel 
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7g.  Tool  room  personnel 


7h.  CAD-CAM  designers 


7i.  Design  engineers 


7j.  Process  planning  engineers 


7k.  Manufacturing  consultants 


71.  Plant  managers 


7m.  Schedulers 


7n.  Schedulers 


7o.  Accountants 


7p.  Management  information  systems  personnel 


7q.  Business  consultants 


7r.  External  environment  personnel 


7s.  Other  (please  specify) 


FINAL  REPORT 


75 


8.  Are  you  familiar  with  interactive  text-based  virtual  reality 
information  presentations  (i.e.,  on-line  dungeons  and  dragons 
games)?  (Check  one) 

(a)  Yes  [ ] (b)  No  [ ] (c)  Somewhat  [ ] 

Comments  (if  any) : 


9.  When  will  text-based  information  presentations  using  VR- 
computer-interf ace  technology  first  be  used  in  manufacturing? 
(Please  indicate  with  an  ”x*'  on  the  scale)  : 

/ / / / 

1995  2005  2015  2025  >2025 

Comments  (if  any) : 


10.  Are  you  familiar  with  interactive  graphics-based  virtual 
reality  information  presentations  (head-mounted  displays,  small 
or  large  screens  in  entertainment)?  (Check  one) 

(a)  Yes  [ ] (b)  No  [ ] (c)  Somewhat  [ ] 

Comments  (if  any) : 


11.  When  will  graphics-based  information  presentations  on  VR- 
computer-interface  technology  first  be  of  used  in  manufacturing? 
(Please  indicate  with  an  ”x”  on  the  scale) : 

/ / / / 

1995  2005  2015  2025  >2025 

Comments  (if  any)  : 


12.  What  do  you  believe  are  the  most  important  features  of 
information  presentations  with  VR  computer-interface  technology 
for  manufacturing? 
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13.  With  respect  to  VR  graphic  presentations,  which  features  are 
most  important  to  you:  (Please  prioritize;  Most  important  = 1) 

(a)  Visual  quality  [ ] (b)  Refresh  speed  [ ] (c)  Seamless  [ ] 

(d)  Tactile  interactions  [ ] (e)  Other  (specify  below)  [ ] 

Comments  (if  any) : 


14.  What  do  you  think  of  the  likelihood  that  information 
structures  and  VR  user  interfaces,  customized  to  support  specific 
users  will  be  useful  and  valuable  in  manufacturing; 

/ / / / / / 

P=0  .2  .4  .6  .8  1.0 

Low  Moderate  High  Certain 

Probability  Probability  Probability 

Comments  (if  any) ; 


15.  What  would  be  the  value  added  by  customized  information 
presentations  on  VR  computer-interface  technology  for 
manufacturing?  (Please  indicate  with  an  "x”  on  the  scale) ; 

/ / / / / / 

0 1 2 3 4 5 

None  Moderate  High 

Comments  (if  any) ; 


16.  Who  will  drive  information  presentations  and  VR  computer- 
interface  technology  for  manufacturing?  (Check  one) 

(a)  Developers  [ ] (b)  Users  [ ] (c)  Both  [ ] 

Comments  (if  any) : 
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17.  Who  will  be  primary  funding  sources  for  information 
presentations  with  VR  computer-interface  technology  for 
manufacturing?  (Check  one) 

(a)  Developers  [ ] (b)  Users  [ ] (c)  Government  [ ] 

Comments  (if  any)  : 


18.  Which  technologies  do  you  believe  must  be  exploited  to  make 
information  presentations  with  VR  computer-interface  technology 
for  manufacturing  workable  and  affordable  to  users? 


19.  Which  industries  or  types  of  processes  do  you  feel  can 
benefit  first  from  information  presentations  with  VR  computer 
interface  technology  for  manufacturing  applications? 


20.  How  would  you  rank  the  necessary  growth  of  the  following 
technological  factors  for  the  integration  of  information 
presentations  on  VR  computer-interface  technology  into 
manufacturing?  (Please  indicate  with  an  **x**  on  the  scales  for 
each  factor) : 

20a.  Computer  hardware?  / — / / / / / 

0 1 2 3 4 5 

None  A Great  Deal 

20b.  Communications  systems?  / / / / / / 

0 1 2 3 4 5 

None  A Great  Deal 

20c.  Software?  / / / / / / 

0 1 2 3 4 5 

None  A Great  Deal 

20d.  Human  Factors?  / / / / / / 

0 1 2 3 4 5 

None  A Great  Deal 

Comments  (in  any): 
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21.  How  well  can  standards,  open  architectures,  and 
interoperability  be  achieved  in  information  presentations  with  VR 
computer-interface  technology  to  define  a framework  across 
multiple  vendors  of  hardware  and  software  and  users?  (Please 
indicate  with  an  **x**  on  the  scale)  : 

/ / / / / / 

0 1 2 3 4 5 

Poor  Excellent 

Comments  (if  any) : 


22.  Does  special  equipment  now  required  for  realism  - head 
mounted  displays,  data  gloves  - reduce  or  enhance  the  ability  of 
the  user?  (Please  indicate  with  an  "x"  on  the  scale) : 

/ / / / / / 

0 1 2 3 4 5 

Reduce  Enhance 

Comments  (if  any) : 


23.  How  well  can  VR  tools  for  the  information  presentation  with 
VR  computer-interface  technology  be  integrated  with  manufacturing 
technology  tools?  (Please  indicate  with  an  "x”  on  the  scale) : 

/ / / / / / 

0 1 2 3 4 5 

Poor  Very  Well 

Comments  (if  any): 


24.  How  would  you  rate  the  applicability  of  the  fidelity  of 
information  presentation  achieved  with  VR  computer-interface 
technology  in  manufacturing  technology  for  discrete  part 
production  in  each  of  the  following  areas?  (Please  indicate  with 
an  “x”  on  the  scales) : 

24a.  Training?  / / / / / / 

0 1 2 3 4 5 

Poor  Excellent 

24b.  Product  design?  / / / / / / 

0 1 2 3 4 5 

Poor  Excellent 
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24c.  Process  design? 


24d.  Process  validation? 


24e.  Enterprise  integration? 


Comments  (in  any) : 


/ / / / 

0 12  3 

Poor 

/ / / / 

0 12  3 

Poor 

/ / / / 

0 12  3 

Poor 


/ / 

4 5 

Excellent 

/ / 

4 5 

Excellent 

/ / 

4 5 

Excellent 


25.  Can  the  US  manufacturing  industry  afford  to  apply  high- 
fidelity,  customized  information  presentations  with  VR  computer- 
interface  technology?  (Please  indicate  with  an  "x”  on  the 
scale) : 


/ / / / / / 

0 1 2 3 4 5 

Can  not  Can 

Comments  (in  any) : 


26.  How  much  will  cost  of  equipment  to  support  a high  quality 
graphical  and  interaction  interfaces  (future  generations  of 
workstations,  head-mounted  displays,  and  data  gloves)  effect 
acceptance  of  a virtual  workplace?  (Please  indicate  with  an 
**x*'  on  the  scale)  : 

/ / / / / / 

0 1 2 3 4 5 

None  A Great  Deal 

Comments  (in  any): 


27.  Do  you  know  of  direct  requirements  for  VR  interface 
presentations  for  any  manufacturing  systems?  (Please  check  one) 

(a)  Yes  [ ] (b)  No  [ ] (c)  Maybe  [ ] 

Comments  (if  any)  : 


FINAL  REPORT 


80 


28.  Will  information  presentation  with  VR  computer-interface 
technology  have  an  overall  worth  in  integrating  companies, 
enterprises,  or  industries  more  intelligently  into  the 
manufacturing  process?  (check  one) 

(a)  Yes  [ ] (b)  No  [ ] (c)  Maybe  [ ] 

Comments  (if  any) : 


29.  Will  information  presentation  with  VR  computer-interface 
technology  integrate  vendors/suppliers  to  a greater  extent  into 
the  manufacturing  process?  (Please  check  one) 

(a)  Yes  [ ] (b)  No  [ ] (c)  Maybe  [ ] 

Comments  (if  any) : 


30.  Please  make  any  additional  comments  below: 
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APPENDIX  IV 


DR.  ERNIE  KENT: 


CHARTS 


1.  Why  is  NIST  sponsoring  this  workshop? 

2.  Why  am  I here? 


3.  What’s  in  it  for  me? 


* NIST  perceives  an  evolving  interest  in  the  intersection  of  a new  technology 
with  manufacturing/industrial  methods. 
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The  design  of  new  mail  processing  facilities  and  the  upgrade  of  old  facilities  are  complex  tasks 
requiring  extensive  analysis.  To  achieve  the  desired  mail  flow  rates,  the  analysis  must  address 
overall  system  layout,  mechanization  requirements,  and  associated  economic  factors.  The 
difficulty  of  understanding  what  effects  changes  made  within  one  part  of  the  system  will  have  on 
the  performance  of  the  system  as  a whole  gives  rise  to  the  need  for  an  integrated  design,  analysis, 
and  simulation  tool  that  will  supply  these  capabilities. 

This  paper  describes  a software  product  called  the  Control  and  Simulation  Analysis  Tool 
(CSAT)  that  assists  in  the  design,  analysis,  and  control  of  mail  processing  facilities.  CSAT  creates 
a very  detailed,  high  fidelity,  three-dimensional  graphic  representation  of  a facility  that  helps 
the  designer  understand  and  take  advantage  of  the  interdependence  among  processing,  staging,  and 
transport  systems,  and  test  different  configurations  and  combinations  of  each  system  under  a 
variety  of  mail  volumes  and  characteristics.  Not  a traditional  simulation,  CSAT  is  rather  a real- 
time control  system  developed  using  a task  decomposition  methodology  that  provides  a high 
degree  of  fidelity  and  realism. 


INTRODUCTION 

The  United  States  Postal  Service,  as  part 
of  its  corporate  goals,  has  made  a commit- 
ment to  automate  the  processing  of  all  types 
and  classes  of  mail.  TTiis  automation  is  cur- 
rently accomplished  through  the  use  of 
facer-cancelers,  optical  character  readers,  and 
bar  code  sorters.  These  high-speed  process- 
ing activities  must  be  supported  with  high 
speed  material  handling  systems  in  order  to 
achieve  the  desired  throughput  rates.  The 
USPS  is  interested  in  methods  to  more  accu- 
rately simulate  large  mail  processing  facili- 
ties, as  well  as  ways  to  analyze  the  effects  of 
introducing  new  material  handling  systems 
and  re-configuring  existing  ones. 

The  Control  and  Simulation  Analysis 
Tool  (CSAT)  is  a multi-functional  computer 
animation  and  device  emulation  program, 
capable  of  modeling,  in  real  time,  material 


handling  systems  and  mail  processing 
equipment.  It  is  able  to  emulate  the  behav- 
ior of  mail  processing  machines  that  exist  in 
the  Postal  ^rvice  today,  as  well  as  systems 
that  are  envisioned  in  the  future,  and  link 
them  using  a common  software  architecture 
to  create  an  integrated  system  that  functions 
as  a unified  whole. 

TRADITIONAL  SIMULATIONS 

Virtually  all  computer  programs  that 
simulate  the  behavior  of  equipment  employ 
methods  borrowed  from  operations  research 
and  computer-aided  design  and  manufac- 
turing (CAD /CAM). 

Simulations  that  use  these  traditional 
methods  are  extremely  important.  They  can 
be  used  to  provide  a wealth  of  information 
to  feed  initial  estimates  relating  to  system 


capacity,  overall  processing  requirements, 
and  equipment  performance  values. 

However,  traditional  simulations  are 
unable  to  form  the  basis  for  actual  control 
systems.  Since,  by  their  nature,  they  embed 
control  assumptions  within  their  simula- 
tions, it  is  generally  not  possible  for  to  sepa- 
rate what  is  control  from  what  is  simula- 
tion. 

These  programs  combine  control  logic 
with  statistical  approximations  of  behavior 
in  a manner  that  makes  it  very  difficult  to 
either  determine  and  understand  the  con- 
trol issues,  or  to  independently  verify  per- 
formance values. 

THE  RCS  METHODOLOGY 

Initially  developed  at  the  National 
Institute  of  Standards  & Technology  (NIST), 
the  Real-time  Control  System  (RCS)  archi- 
tecture forms  the  foundation  of  the  CSAT 
product.  Now  being  promoted  by  NIST  as  a 
standard  for  real-time  control  software,  the 
RCS  methodology  has  evolved  for  over  15 
years  to  become  a viable  system  for  generic 
real-time  control  of  complex  systems. 

The  RCS  methodology  prescribes  a soft- 
ware framework  requiring  all  real-time  con- 
trol systems  to  be  modeled  as  hierarchically 
structured  organizations  of  layered  respon- 
sibilities. It  further  requires  that  these  layers 
be  built  from  generic  control  module  struc- 
tures as  unit  building  blocks. 

These  generic  control  modules  are 
designed  in  such  a way  that  the  lowest  level 
modules  correspond  exactly  to  the  actual 
hardware  devices  that  will  be  controlled  in  a 
real  system.  Connected  to  these  device  con- 
trollers in  a one-to-one  relationship  are 
mechanism  simulators  that  emulate  the 
behavior  of  real  devices.  An  environment 
simulation  module  combines  the  low-level 
device  operations  (such  as  conveyors,  pho- 


tocells, and  diverters)  with  physical  charac- 
teristics of  objects  (such  as  trays  and  sacks  of 
mail)  in  the  environment  to  determine  how 
these  objects  will  move  in  response  to 
device  movement. 

The  combination  of  both  simulations 
(mechanism  and  environment)  creates  a 
model  of  the  behavior  of  the  hardware  sys- 
tem that  is  separate  from  the  control  soft- 
ware which  resides  in  the  upper  layers  of  the 
RCS  hierarchy. 


Because  of  this  clean  separation  between 
control  logic  and  simulation  software,  the 
control  software,  with  very  little  modifi- 
cation, can  be  connected  to  real  hardware 
immediately,  and  with  a degree  of  confi- 
dence that  is  not  possible  with  traditional 
approaches  to  simulation. 

CSAT  CAPABILITIES 

Although  it  is  designed  to  be  an  inte- 
grated and  comprehensive  solution  to  a 
variety  of  problems,  the  general  functions  of 
the  CSAT  program  can  be  broken  into  eight 
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separate  areas,  each  providing  a unique  set 
of  capabilities. 

3-D  Walk-Through 

A real-time,  three-dimensional  "walk 
through"  of  a mail  processing  facility  is  pro- 
vided as  a standard  capability  through  the 
use  of  a Silicon  Graphics  computer  with  a 
high-resolution  color  display.  All  mail  pro- 
cessing equipment  is  represented,  as  well  as 
all  material  handling  components, 
including  conveyors,  self-guided  vehicles, 
tray  management  systems,  and  automatic 
storage  and  retrieval  systems.  All  relevant 
building  structures  are  visible,  and  may  be 
viewed  from  any  angle  and  from  any 
distance. 

Equipment  Emulation 

All  mail  processing  machines,  as  well  as 
the  material  handling  equipment  and 
people  that  support  and  integrate  them,  are 
emulated  in  real  time.  The  actual  sort  algo- 
rithms that  are  used  by  the  mail  processing 
machines  are  executed,  using  a represen- 
tative database  of  individual  mail  pieces. 
Mail  flow  rates  and  mail  arrival  profiles 
may  be  modified  by  the  operator  to  generate 
detailed  analyses  on  machine  and  material 
handling  performance  based  on  different 
input  conditions. 

Letter  Processing 

The  entire  letter  processing  path  of  a 
mail  processing  facility  is  emulated.  The 
capability  to  move  hardware  and  reconfig- 
ure the  system  is  available.  Modeled  hard- 
ware includes  such  components  as  staging 
towers,  monorails,  conveyors,  vertical  lifts, 
and  Postal  Automated  Staging  and  Retrieval 
Systems,  as  well  as  letter  sorting  machines. 
Each  device  has  associated  attributes  includ- 
ing mail  processing  rates.  The  operator  has 
the  ability  to  choose  from  among  a variety  of 
devices  in  order  to  configure  a complete  sys- 
tem. The  tool  allows  re-configuration  of  the 
machine  layout,  showing  the  effects  of  mail 
flow  for  each  given  sort  scheme. 


Flats  & Parcels 

The  capability  to  process  "flats",  small 
parcels,  and  bundles  is  also  provided. 
Mechanism  simulators  include  flat  sorters, 
small  parcel  and  bundle  sorters,  sack  and 
tray  sorters,  and  automatic  loader-unloaders. 

Coordinated  Control 

One  of  the  most  difficult  aspects  of  inte- 
grating real-time  control  and  simulation 
software  is  the  coordination  and  integration 
of  all  the  various  devices  used  in  a mail  pro- 
cessing facility.  In  order  to  achieve  opti- 
mum mail  processing  rates,  the  ability  to 
experiment  with  different  transportation 
schedules,  both  into  and  out  of  the  mail  pro- 
cessing facility  is  provided,  as  well  as  the 
ability  to  control  groups  of  heterogeneous, 
cooperating  machines. 

CAD  Input  & Report  Output 

Layout  descriptions  of  equipment  gener- 
ated from  CAD  drawings  can  be  brought  in 
through  numerical  files  and  interpreted  by 
CSAT.  Furthermore,  reports  are  generated 
that  outline  any  information  requested  by 
the  operator,  such  as  number  of  trays  of  a 
particular  zip  code  and  mail  processing  rates 
for  a given  piece  of  hardware. 

Cost  Estimation 

A module  has  been  added  to  CSAT  to 
provide  complete  pricing  and  parts  count 
information  for  each  given  configuration. 
Outputs  from  this  module  are  displayed  on 
the  screen  or  included  in  output  reports. 

Operator  Interface  Module 

CSAT  is  being  designed  to  be  as  user- 
friendly  as  possible,  and  allows  the  op>erator 
to  use  common  input  devices  to  configure 
the  system,  change  operating/simulation 
parameters,  request  analysis  and  displays  of 
data,  request  present  operating  status  and 
inventories,  and  enter  data  and  operational 
commands.  CSAT  processes  information 
for  display  to  the  operator,  including 
selected  sort  plans  and  currently  activated 
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mechanisms.  It  also  retrieves  and  displays 
such  information  from  hardware  controllers 
and  simulators  as  the  number  of  trays  of 
mail  destined  for  delivery  to  a particular  zip 
code. 

CONCLUSION 

CSAT  is  being  designed  to  interface  with 
real-time  hardware  systems,  bringing  in  sen- 
sor and  actuator  values  in  real  time  so  that 
the  display  becomes  the  operator's  display  of 
the  actual  operating  facility.  This  reuse  of 
the  display  capability  saves  a considerable 
cost  in  the  development  of  the  operator's 
display  for  the  running  systems.  Operator 
input  devices  such  as  touch  screens  will 
allow  the  operator  to  point  at  a component, 
determine  its  status,  direct  its  operation,  and 
check  its  maintenance  schedule. 

All  simulation/animation  systems  must 
address  the  issues  of  accuracy  and  believabil- 
ity.  Because  the  CSAT  software  is  written  to 
control  and  emulate  actual  hardware 
devices  down  to  the  motor/ actuator  level, 
the  issue  of  accuracy  is  no  longer  a question 
of  the  assumptions  used  to  estimate  perfor- 
mance of  high-level  machine  behavior,  but 
rather  of  the  performance  characteristics  of 
the  machines  themselves.  This  translates 
directly  into  more  believable  simulation 
results. 
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APPENDIX  VII 


DR.  JAMES  6EIWITZ: 


CHARTS 


OBJECTIVES 


STANDARDIZATION  OF  PROTOCOLS 

FOR  THE  APPLICATION  OF  INFORMATION  AND  INTERFACE 
TECHNOLOGY 

FOR  VIRTUAL  AND  DISTRIBUTED  MANUFACTURING  OF 
DISCRETE  PARTS 


INTEGRATION  OF  HUMAN  FACTORS 
VIRTUAL  REALITY 
MANUFACTURING  SYSTEMS 


VISION  NEEDED: 

HUMAN  FACTORS  IN  INFORMATION  AND  INTERFACES 
NEEDED  TO  INTEGRATE  VIRTUAL  REALITY  TECHNOLOGIES 
INTO  THE  VIRTUAL  AND  DISTRIBUTED  MANUFACTURING 
OF  DISCRETE  PARTS 


PARTICIPANTS  WILL  ADDRESS: 

1.  DEFINITION  OF  REQUISITE  M^FQRMATION  AND 
INTERFACES  FOR  VIRTUAL  AND  DISTRIBUTED 
MANUFACTURING  APPLICATIONS 

2.  HUMAN  FACTORS  APPROACHES  TO  INFORMATION  AND 
INTERFACE  REQUIREMENtS: 

3.  LEVERAGING  OF  EFFORTS  IN  SPECIFIC  TOPIC  AREAS 


VIRTUAL  REALITY 


WHAT  IS  VIRTUAL  REALITY? 

WHAT  IS  THE  DIFFERENCE  BETWEEN  VIRTUAL  REALITY  AND 

- ARTinCIAL  WORLDS? 

- SYNTHETIC  ENVIRONMENTS? 

- TELEOPERATION? 

- TELEVISED  ENVIRONMENT? 

VIRTUAL  REALITIES  LIE  ON  A DIMENSION 

BUT  WHAT  IS  THE  DIMENSION? 

- SIMILARITY  TO  REAL  WORLD? 

~ NUMBER  OF  VIRTUAL  FEATURES?  MODES? 

~ AMBITION! 


not 

ambitious 


ambitious 


text'based 
interactions 
(Dungeons  & Dragons) 


the  illusion  of  a real  world: 
immersed  in  a 
synthetic  or  remote 
environment 


graphics'based 

interactions 

(interactions  w.  objects) 


VIRTUAL  REALITY  = 

THE  TRUE-TO-FORM  REPRESENTATION  OF  THE  OBJECTS  IN 
THE  REAL  WORLD  AND  THEIR  INTERACTIONS 

TELEVISION  AND  TELEROBOTICS:  PRESENTATION  OF  THE  REAL 
WORLD,  NOT  REPRESENTATION 

ARTIFICIAL  WORLDS:  NOT  NECESSARILY  CONSTRAINED  BY 
REAL-WORLD  LIMITATIONS,  e.g.,  THE  LAWS  OF  PHYSICS 


MODES  OF  VIRTUAL  REALITY 


I.  CONCEPTUAL  (e.g.,  text*based  VR) 

II.  SENSORY: 

VISUAL  (e.g.,  SIMNET) 

AUDITORY  (e.g.,  LOOMIS/KLATZKY  NAVIGATIONAL  AID) 
HAPTIC  (e.g.,  SENSING  ROBOTS) 

TASTE  & SMELL  (e.g.,  SCRATCH  & SNIFF  MOVIES) 


III.  MOTOR  (interactions  with  the  virtual  environment) 


MANUFACTURING  SYSTEMS 


A (USUALLY  AUTOMATED)  DEVICE  OR  SET  OF  DEVICES 
THAT  AIDS  IN  SOME  ASPECT  OF  THE  MANUFACTURING 

PROCESS  (e.g.,  REACTOR  PROTECTION  SYSTEM  (process  control), 
CAD-CAM) 


MANUFACTURING  AS  A FEEDBACK  SYSTEM 
(after  Helander,  1993) 

TOP-DOWN  BOTTOM-UP 

PLANNING  EVALUATION 


PRIMARY  TARGET  AREAS  (from  invitation); 

ENGINEERING  DESIGN  PRODUCT  PLANNING 

PROCESS  PLANNING  FACTORY  SIMULATION 

MONITORING  & CONTROL 


THE  HUMAN-FACTORS  ISSUES 


WHAT  DOES  THE  USER  NEED  TO  KNOW 
TO  PERFORM  THE  TASK? 


WHEN  IS  THE  KNOWLEDGE  NEEDED? 

I 


IN  WHAT  FORM  IS  the  knowledge 
MOST  conducive  TO 
EFFECTIVE  USE? 


A FEW  RELEVANT  RESEARCH  STUDIES 


1.  GENERAL  MOTORS  (1980s);  INVESTED  $80  BILLION  IN 
AUTOMATED  MANUFACTURING;  $20  BILUON  DIDNT  WORK 

— main  problems:  poor  task  analysis,  poor  functional  allocation, 
inadequate  training 

2.  Helander  & Furtado,  1992:  ROBOTS  FOR  PRODUCT  ASSEMBLY; 
ROBOTS  w.  NO  VISION  AND  POOR  HNGER  DEXTERITY  (too 
expensive),  SO  PRODUCT  HAD  TO  BE  REDESIGNED  FOR  EASY 
ASSEMBLY  (e.g.,  "use  snap  and  insert  assembly"). 

RESULTS:  CHANGES  MADE  MANUAL  ASSEMBLY  EASIER  TOO, 
TO  THE  EXTENT  THAT  IT  WAS  CHEAPER  TO  USE  MANUAL 
LABORI 


3.  CONCURRENT  ENGINEERING:  PRODUCT  DESIGNERS  & 
PROCESS  DESIGNERS  WORKING  TC)GETHER  TO  DESIGN  A 
PRODUCT  THAT  CAN  BE  MANUFACTURED  BY  CERTAIN 
STANDARDS:  ' 

- EASE  OF  ASSEMBLY 

- DESIGN  FOR  MAINTAINABILITY 

- INTEROPERABILTY 

- WORKER  AND  USER  SAFETY 


4.  COMPUTER-INTEGRATED  MANUFACTURING  (CIM): 
Siemiemiuch  (1992):  9 million  pounds  sterling,  CIM  project,  TOTAL 
FAILURE  * , 


— main  problems;  poor  Usk  analysis,  poor  knowledge  acquisition, 
poor  pre-implementation' validation 


5.  NO  MAJOR  FIELp  STUDIES  OF  AUTOiyiATED 
MANUFACTURING;  IN  FACT, THE  LA§T  MAJOR  FIELD  STUDY  OF 
ANY  KIND  OF  MANUFACTURING  PROCESS  WAS  1935,  THE 
HAWTHORNE  STUei|S>T  WESTERN  ELECTRIC 

— improved  lighting  jmgroyed  productivity,  BUT 

— degraded  lighting  (returlied  to  normai)  also  improved 

productivity  ‘ | ^ : 

— workers  who  got  attention  froin  supemsors  (lighting  changes) 
responded  with  increased  production  (ttie  Hawthorne  Effect) 

f = ■ I I 1 ! : i ■ 


TARGET  TASKS 


1.  INVITATION; 


2.  TEAM  TASKS: 


ENGINEERING  DESIGN, 
PRODUCT  PLANNING, 
PROCESS  PLANNING, 
FACTORY  SIMULATION, 
MONITORING  & CONTROL 


COMPUTER-SUPPORTED  COOPERATIVE 
WORK 

TELECONFERENCING  IN  VIRTUAL 
ENVIRONMENTS 

NETWORKING  IN  VIRTUAL  ENVIRONMENTS 
DISTRIBUTED  DECISION  MAKING 
CONCURRENT  ENGINEERING 

t j ; ? . 


MAJOR  HUMAN  FACTORS  ISSUES 


1.  GROUP  COGNITIVE  TASK  ANALYSIS 

- FUNCTION  ALLOCATION 

- COORDINATION 

- SYNCHRONIZATION 


2.  KNOWLEDGE  ACQUISITION 

- SEMANTIC  & PROCEDURAL  KNOWLEDGE 

- EXPERT  MAY  NOT  BE  ABLE  TO  ARTICULATE  KNOWLEDGE 

- KNOWLEDGE  ACQUISITION  GUIDEBOOK  (55  KATs) 

• » 


3.  NEUROPSYCHOLOGICAL  STUDIES  OF  PERCEPTION 

- VISION 

— AUDITION  (voice  recognition) 

- HAPTICS  (data  glove,  finger  exoskeleton) 

- MOTOR  BEHAVIOR  ‘ 


4.  MODELS  OF  THE  MANUFACTURING  PROCESS 
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5.  MODELS  OF  AUTONOMOUS,  INTERACTIVE  AGENTS 
- SIMNET  & V(INT)2:  playing  the  OPFOR 
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APPENDIX  VIII 


DR. 


JAMES  6EIWITZ:  POST  WORKSHOP  COMMENTS 


HUMAN  FACTORS  ISSUES  IN  VIRTUAL  REALITY  INTERFACES 

FOR  MANUFACTURING 

A Report  on  the  NIST  Workshop,  August  9,  1994 

By 

James  Geivitz,  Carnegie  Mellon  Research  Institute 

for 

Ernest  W.  Kent,  National  Institute  of  Standards  and  Technology 


1.  The  purpose  of  this  report  is  to  summarize  the  human  factors 
that  were  raised  during  the  day-long  workshop  on  virtual  reality 
interfaces  for  manufacturing.  These  issues  were  raised  during 
morning  presentations  and  their  discussions  or  during  the 
afternoon  breakout  sessions.  The  objective  of  the  workshop  was 
the  integration  of  three  areas  of  R&D:  human  factors,  virtual 
reality  technology,  and  manufacturing  systems. 

2.  In  Dr.  Geiwitz's  presentation,  he  identified  the  three  basic 
human  factors  issues:  What  does  the  user  need  to  know  to  perform 
the  task?  When  is  the  knowledge  required?  In  what  form  is  the 
knowledge  most  conducive  to  effective  use?  Dr.  Harold  Van  Cott, 

a prominent  human-factors  psychologist,  suggested  adding  a "basic 
issue"  to  cover  the  testing  and  evaluation  of  the  VR  interface: 
Compared  to  conventional  methods  of  manufacturing,  is  the  VR 
interface  more  effective  and/or  more  efficient?  We  might  also 
add  a determination  of  cost  effectiveness,  that  is,  is  the  higher 
cost  of  VR  interfaces  worth  the  increase  in  performance? 

3.  The  issues  in  #2  depend  on  a thorough  task  analysis.  A task 
analysis  not  only  describes  the  steps  in  task  performance,  it 
analyzes  them  for  the  knowledge  and  skills  required  to  perform 
the  task  proficiently.  In  this  case,  we  need  a cognitive  task 
analysis,  because  some  of  the  steps  in  task  performance  will  be 
mental  — a decision,  an  inspection.  Cognitive  task  analysis 
uses  the  techniques  of  knowledge  acquisition  to  analyze  the 
knowledge-skill  requirements.  In  distributed  manufacturing 
tasks,  a group-cognitive-task-analysis  (GCTA)  is  required,  in 
which  the  steps  performed  by  different  individuals  (at  possibly 
different  locations)  are  analyzed;  the  group  task,  such  as 
communication,  coordination,  and  synchronization,  must  also  be 
analyzed.  A methodology  for  GCTA  has  not  yet  been  developed, 
although  Ed  Salas  of  ONR  has  been  working  on  it  for  team  training 
purposes.  Jim  Geiwitz  has  developed  the  theoretical  basis  for 
GCTA  and  is  presently  developing  the  methodology. 

4.  In  addition  to  the  task  analysis,  an  information  needs 
analysis  is  required,  to  answer  the  three  basic  human  factors 
questions  described  in  #2. 

5.  An  important  question  that  surfaced  quickly  is.  What  is 
virtual  reality?  Dr.  Loftin  defined  it  as,  "To  some  degree. 
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occupying  an  environment  other  than  the  physical  environment  of 
the  moment.*'  Dr.  Geiwitz  defined  it  as,  "The  true-to-form 
representation  of  the  objects  in  the  real  world  and  their 
interactions."  The  participants  discussed  the  differences 
between  VR  and  various  other  representations;  artificial  worlds, 
synthetic  environments,  teleoperation,  a televised  environment, 
augmented  reality  (graphics  superimposed  on  the  real  world) . 

Most  preferred  "synthetic  environment"  to  VR  environment.  An 
artificial  world  is  not  necessarily  VR;  it  may  relieve  the 
constraints  of  the  real  world,  such  as  the  laws  of  physics. 
Augmented  reality  raises  some  interesting  human-factors  issues  of 
its  own.  For  example,  as  shown  by  Dan  Weintraub  at  NASA-Ames,  a 
VR  graphic  of  the  landing  strip  can  be  presented  to  a pilot,  an 
immense  aid  to  the  pilot's  approach  before  he  or  she  can  see  the 
strip  visually.  When  the  pilot  breaks  through  the  clouds  and  can 
see  the  strip,  the  graphic  remains  as  augmented  reality, 
superimposed  on  the  real  runway.  The  problem  is,  the  graphic  may 
interfere  with  normal  vision,  with  potentially  disastrous 
consequences:  Weintraub  was  able  to  show  that  if  the  real  runway 

had  a plane  parked  on  it,  an  extremely  dangerous  landing 
situation,  the  pilot  could  not  perceive  the  other  plane  until  too 
late  to  avoid  a crash.  The  VR  graphic  dominated  the  pilots 's 
perception,  at  the  expense  of  the  perception  of  the  real  world. 

6.  Is  this  a VR  experience?  I don't  drink,  so  when  I join 
friends  at  a bar,  I order  a nonalcoholic  beer,  such  as  O'Douls. 

On  one  occasion,  another  friend  joined  us  late.  She  saw  my  beer 
and  ordered  one  for  herself.  She  did  not  know  it  contained  no 
alcohol.  After  4 or  5 O'Douls,  she  got  drunk,  slurring  her 
words,  stumbling  around,  and  laughing  inappropriately  at  bad 
jokes.  Was  she  in  a VR  environment  created  by  her  own 
imagination?  Certainly  she  was  occupying  an  environment  other 
than  the  physical  environment  of  the  moment.  Food  for  thought. 

7.  Dr.  Geiwitz  brought  up  the  question  of  a dimension  on  which 
VR  environments  lie.  Everyone  seems  to  agree  that  such  a 
dimension  exists,  but  the  participants  argued  about  the  nature  of 
the  dimension.  What  is  it  a dimension  of?  The  background 
material  for  the  workshop  called  it  a dimension  of  "ambition." 

At  the  ambitious  end  of  the  dimension,  the  illusion  of  a real 
worlds  is  created  and  the  user  is  immersed  in  a synthetic  or 
remote  environment.  At  the  not  ambitious  end  of  the  dimension 
are  such  environments  as  those  created  by  text  alone  (as  in 
Dungeons  and  Dragons)  or  simple  graphics  representing  the  real 
world  (I  suppose  Weintraub 's  VR  runaway  would  be  an  example) . 

Dr.  Loftin  called  it  a dimension  of  fidelity,  that  is,  how  exact 
is  the  mapping  of  real  world  onto  the  VR  world?  Fidelity  has  at 
least  two  components:  1)  stimulus  similarity  and  2)  the  number 
of  modes  used  and  represented.  Presumably,  a VR  environment  with 
visual,  auditory,  and  haptic  displays  has  more  fidelity  than  one 
with  only  visual  displays.  The  importance  of  this  dimension, 
whatever  one  calls  it,  is  that  one  would  like  to  create  just 
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enough  VR  to  improve  performance;  any  more  than  this  is  costly 
and  unnecessary.  Slight  advances  in  fidelity  may  not  improve 
performance  enough  to  balance  the  increased  cost.  The  human- 
factors  issue  here,  as  stated  by  Dr.  Loftin,  is  the  determination 
of  the  required  degree  of  fidelity  — a very  difficult  task. 

Several  participants  returned  to  the  idea  of  a dimension  later  in 
the  workshop.  Some  of  the  proposed  dimensions  were  design  vs 
manufacture,  training  vs  manufacturing,  teleoperation  vs  direct 
operation,  controller  end  vs  controllee  end,  distant  in  time  vs 
distant  in  space,  engagement  vs  immersion,  dynamic  vs  static, 
interactive  vs  no  interactive. 

8.  Dr.  Geiwitz  identified  three  primary  modes  of  VR:  conceptual 
(e.g.,  text-based  VR) , sensory  (visual,  auditory,  haptic, 
olfactory,  and  gustatory),  and  motor  (interactions  with  the  VR 
environment) . Most  VR  today  is  visual,  with  auditory  a distant 
second.  Dr.  Loftin  identified  haptic  displays  as  an  important 
area  of  VR  research,  and  Dr.  Stansfield  (Sandia)  is  about  to 
begin  a research  program  on  haptic  VR.  In  nuclear  power  plants 
where  much  of  process  occurs  in  the  dark  or  under  water,  haptic 
displays  may  be  more  important  than  visual  or  auditory  ones, 
especially  for  maintenance  activities. 

9.  Dr.  Geiwitz  reviewed  a few  relevant  research  studies,  which 
indicate  some  of  the  problems  that  the  implementation  of  VR 
environments  for  manufacturing  are  likely  to  have.  In  almost  all 
cases,  inadequate  task  analysis  was  the  fundamental  cause  of  the 
failure  of  VR  and  other  automated  manufacturing  systems  (e.g., 
CIM) . Poor  knowledge  acquisition  was  another  common  problem. 
Other  problems  included  poor  functional  allocation  (allocating 
subtasks  to  humans  or  machines) , inadequate  training  for  the  use 
of  the  systems,  and  poor  pre-implementation  validation.  Dr. 
Loftin  noted  a number  of  technical  problems  that  arise  with  VR; 
Eye  tracking  devices  do  not  work  very  well  (yet) ; nausea, 
headaches,  and  eye  strain  are  common  among  VR  users;  and, 
finally,  won't  somebody  invent  a good  3-D  mouse  to  facilitate 
interaction? 

10.  Research  has  discovered  some  interesting  aspects  of  VR 
development.  For  example,  in  one  study  of  the  use  of  robots  for 
object  assembly,  the  robots  had  on  vision  and  poor  finger 
dexterity  (cost  considerations) . The  product  therefore  had  to  be 
redesigned  for  easy  robot  assembly,  e.g.,  "use  snap  and  insert 
assemblies,  wherever  possible.”  These  changes,  however,  made 
manual  assembly  easier,  too,  to  the  extent  that  it  became  cheaper 
to  use  manual  labor  for  assembly;  the  robots  were  junked!  On  the 
other  hand,  research  like  this  led  to  the  development  of 
Concurrent  Engineering,  in  which  the  products  designers  and  the 
process  designers  work  together  from  the  beginning,  to  design  a 
product  that  can  be  manufactured  by  certain  standards,  such  as 
ease  of  assembly,  design  for  maintainability,  interoperability. 
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and  worker/user  safety.  (Conventional  engineering  has  product 
design  first,  then  process  design) . 

The  last  major  field  study  of  manufacturing  process  was  the 
Hawthorne  studies  at  Western  Electric  in  1935!  Hawthorne  found 
that  improved  lighting  improved  productivity,  but  degraded 
lighting  (return  to  normal)  also  improved  productivity.  The 
conclusion  was  that  workers  who  got  attention  from  their 
supervisors,  even  if  it  meant  a change  for  the  worse,  responded 
with  increased  production.  This  has  since  been  called  the 
Hawthorne  Effect;  it  must  be  considered  when  making  any  kind  of 
change  that  is  expected  to  increase  performance. 

11.  Clearly,  the  absence  of  manufacturing  engineers  at  the 
workshop  hindered  the  discussion  of  human  factors  issues.  The 
human  factors  engineers  focused  on  the  task:  Task  analysis  is 
the  first  step  in  the  design  of  an  effective  VR  system.  But  what 
is  the  task?  We  could  only  speculate,  and  that  at  a fairly 
generic  level.  Here  are  some  of  the  manufacturing  tasks  to  which 
VR  technology  might  be  applied  (if  the  task  analysis  shows  they 
might  benefit) : engineering  design,  product  planning,  process 
planning,  factory  simulation,  and  monitoring  and  control  (process 
control) . A number  of  group/ team  tasks  were  also  discussed 
during  the  workshop,  including  computer  supported  cooperative 
work,  teleconferencing  in  virtual  environments,  distributed 
decision  making,  and  concurrent  engineering. 

12.  Research  is  needed  on  knowledge  acquisition,  a component  of 
cognitive  task  analysis.  The  expert  has  two  kinds  of  knowledge, 
semantic  (what  to  do)  and  procedural  (how  to  do  it) . Knowledge 
engineers  often  elicit  the  semantic  knowledge  alone,  which  is 
essentially  worthless  without  the  corresponding  procedural 
knowledge.  Experts  often  cannot  articulate  their  procedural 
knowledge  — imagine  you  were  asked  how  to  run  down  a flight  of 
stairs  — and  therefore  ingenious  techniques  must  be  developed  to 
elicit  such  knowledge.  An  example  of  this  problem  and  how  it 
might  be  solved  is  the  college  physics  developed  by  Fred  Reif  and 
Jill  Larkin  at  Carnegie  Mellon.  Students  who  take  a conventional 
physics  course  have  semantic  knowledge  of  physics,  but  they  have 
no  procedural  knowledge.  They  do  not  know  how  to  solve  physics 
problems;  their  average  score  on  the  final  exam  is  around  35% ! 
Reif  and  Larkin  specifically  teach  them  how  to  describe  their 
problem  in  terms  that  afford  a solution,  and  then  teach  them  how 
to  apply  the  principles  and  theories  of  physics  to  solve  the 
problem;  they  teach  procedural  knowledge  to  go  with  the 
principles  and  theories,  the  semantic  knowledge.  Reif  & Larkin's 
students  average  around  85%  correct  on  the  final  exam,  an 
incredible  improvement. 

13.  One  major  problem  in  the  development  of  interactive  VR 
systems  is  how  to  program  autonomous,  interactive  agents  (rather 
than  objects) . To  predict  what  an  agent  will  do  in  any  given 
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situation  is  a difficult  task,  one  that  psychology  has  been 
working  on  for  two  centuries.  Fortunately,  the  range  of  behavior 
in  most  VR  systems  is  limited,  so  the  problem  is  less  than  it 
might  be.  Dr.  Geiwitz  was  involved  in  the  development  of  OPFOR 
agents  in  SIMNET.  In  this  case,  the  domain  of  behavior  was 
combat;  we  had  only  to  develop  a model  of  an  agent  doing  mission 
planning  and  reacting  to  the  combat  maneuvers  of  the  SIMNET  user 
(tank  companies  or  battalions) . 

14.  Some  of  the  applications  of  VR  technology  discussed  during 
the  workshop  include  scientific  data  visualization,  in  which  the 
engineer-designer  navigates  through  large  libraries  of  data,  with 
multisensory  perception,  immersion  for  discovery,  e.g.,  walk 
around  inside  molecules  (Loftin) ; facility  walkthru,  visualize 
maintenance  during  product  design,  object  identification  in 
remote  environments,  determine  wheelchair  access  in  a building 
(Stansfield) ; use  VR  instead  of  full  scale  mockups,  sales  — show 
customer  what  a customized  airplane,  according  to  his  specs  would 
look  like,  feel  like,  try  out  flight  deck  designs  immersed  in  the 
body  of  a 5%  female,  augmented  reality  for  touch-labor  tasks 
(transparent  graphics  on  real  equipment)  (Boeing) ; post  office 
needs  design  to  handle  peak  volume,  identify  bottlenecks  and 
eliminate,  simulations  for  design  and  for  training,  real-time 
situation  awareness:  what  is  happening,  what  has  happened,  what 
will  happen,  estimate  costs  (Clyde) ; several  people  made  the 
point  that  a lot  of  manufacturing  involves  the  need  to  know  state 
of  processing  activity  at  one  point,  to  plan  to  handle  it  at  some 
later  point. 

15.  Some  of  the  applications  discussed  in  the  breakout  sessions 
include  whole  company  use  at  Sikorsky:  design,  manufacturing, 
finance,  sales,  etc.  (problem  is  integration  of  domain  models, 
interactions  among  them;  part  of  the  solution  is  to  have  more 
generalists  and  fewer  specialists) ; in  design,  design  and  test 
(simulation) , to  prevent  errors  in  manufacturing  before  process 
begins;  crane  operator,  needs  to  see  what  he's  doing;  and,  of 
course,  teleoperation. 

16.  The  role  of  reference  models  surfaced  several  times  during 
the  workshop.  The  primary  purpose  of  a reference  model  is  to 
promote  interoperability  or  open  systems.  Typically  these  models 
are  layered,  so  interactions  between  models  can  be  coordinated  at 
several  levels,  from  the  CEO  issuing  a command  to  the  switching 
packets  for  electronic  communication  between  systems,  at  all 
seven  levels.  One  participant  described  her  needs  for  standards 
and  protocols:  standards  for  storage  of  data  for  VR:  CAD — 
>process  planning — > simulation,  pp  also — > scheduling,  all 
arrows  involve  protocols  (sometimes  informal) ; OOP  (object 
oriented  programming)  is  good  for  manufacturing  schedules, 
because  schedules  vary,  but  objects  remain  the  same,  so  do  object 
interactions;  phrased  in  terms  of  responsibilities  — what  has  to 
be  done  — not  when;  OOP  is  good  for  reference  models  as  well,  is 
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what  we  used  for  the  Command  and  Control  Reference  Model  for 
combat  decision  aids. 

17.  Miscellaneous  human-factors  issues:  Several  participants 
commented  on  position  trackers,  don't  work  very  well,  do  not 
locate  well,  also  lag  in  time.  Boom  VR  systems  are  preferred  to 
head  mounts,  because  of  weight.  Voice  commands  are  preferred  to 
data  glove  (Stansfield) . Boeing  representative  urged  caution  on 
generic  test  beds  (but,  in  my  opinion,  the  obstacles  are  no  worse 
than  in  most  complex  domains  (e.g.,  I was  told  that  I could  never 
construct  certification  tests  for  mechanics  in  nuclear  power 
plants  because  every  plant  had  different  equipment  and  different 
procedures,  but  we  did  it,  and  the  general  skills  and  knowledge 
turned  out  to  be  the  most  predictive  of  performance  on  the  job. 

IN  other  words,  I wouldn't  be  concerned  about  the  NIST  testbed; 
if  we  do  it  right,  it  will  facilitate  the  development  of  VR 
systems  for  manufacturing  AND  speed  their  implementation;  I think 
a generic  testbed  at  NIST  is  a wonderful  idea.) 

18.  One  of  the  participants  in  my  breakout  session  drew  a 
diagram  of  how  she  saw  the  VR  systems  work.  I thought  it  was  an 
excellent  diagram  and  therefore  reproduce  it  here  (next  page) . 

She  is  describing  the  construction  of  a graphics  environment,  or 
a VR  graphics  database:  The  designer  builds  a CAD  data  base,  the 
process  planner  builds  process  and  resource  models,  the  scheduler 
develops  a process  plan  and  tests  it  in  simulation  form.  In  the 
bottom  diagram,  the  users  use  intelligent,  domain-specific  tools 
to  interact  with  giant  object  libraries,  called  knowledge  bases 
(KB)  and  data  bases  (DB) . As  she  indicated,  this  is  also 
(roughly)  the  model  used  by  ARPA  in  its  initiative  on  VR  and 
training. 
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